vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #533
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Formatting '/build/vm-state-worker1/tmp.TK9Rze6tJd', fmt=raw size=107374182414worker1 # mke2fs 1.47.4 (6-Mar-2025)15client # Disk image does not exist, creating the virtualisation disk image...16worker1 # Discarding device blocks: 0/262144 done17client: QEMU running (pid 49)18client # Formatting '/build/vm-state-client/tmp.25bTFcZ9qK', fmt=raw size=107374182419worker1: QEMU running (pid 51)20lb # Disk image does not exist, creating the virtualisation disk image...21worker2 # Disk image does not exist, creating the virtualisation disk image...22worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes23lb: QEMU running (pid 50)24client # mke2fs 1.47.4 (6-Mar-2025)25worker2: QEMU running (pid 52)26lb # Formatting '/build/vm-state-lb/tmp.IYbqLiibSh', fmt=raw size=107374182427client # Discarding device blocks: 0/262144 done28worker2 # Formatting '/build/vm-state-worker2/tmp.CdW84R8xjt', fmt=raw size=107374182429worker1 # Filesystem UUID: 07f97f2e-1a27-4afe-86d2-1a582037d5c930(finished: start all VMs, in 0.50 seconds)31worker2 # mke2fs 1.47.4 (6-Mar-2025)32worker1 # Superblock backups stored on blocks:33client: waiting for unit network-addresses-eth1.service34worker2 # Discarding device blocks: 0/262144 done35worker1 # 32768, 98304, 163840, 22937636client: waiting for the VM to finish booting37worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes38worker1 # 39client # Creating filesystem with 262144 4k blocks and 65536 inodes40lb # mke2fs 1.47.4 (6-Mar-2025)41worker2 # Filesystem UUID: 28eb8080-5cf1-4418-abb6-e32a14f1f74c42worker1 # Allocating group tables: 0/8 done43client # Filesystem UUID: c916ece1-5f76-4c92-b04d-2db56ad83b4544lb # Discarding device blocks: 0/262144 done45worker2 # Superblock backups stored on blocks:46worker1 # Writing inode tables: 0/8 done47client # Superblock backups stored on blocks:48lb # Creating filesystem with 262144 4k blocks and 65536 inodes49worker2 # 32768, 98304, 163840, 22937650worker1 # Creating journal (8192 blocks): done51client # 32768, 98304, 163840, 22937652lb # Filesystem UUID: 59281576-d49d-44d1-811e-7aef8f48f1ce53worker2 # 54worker1 # Writing superblocks and filesystem accounting information: 0/8 done55client # 56lb # Superblock backups stored on blocks:57worker2 # Allocating group tables: 0/8 done58worker1 # 59client # Allocating group tables: 0/8 done60lb # 32768, 98304, 163840, 22937661worker2 # Writing inode tables: 0/8 done62worker1 # Virtualisation disk image created.63client # Writing inode tables: 0/8 done64lb # 65worker2 # Creating journal (8192 blocks): done66worker1 # Starting virtiofs daemons...67client # Creating journal (8192 blocks): done68lb # Allocating group tables: 0/8 done69worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70client # Writing superblocks and filesystem accounting information: 0/8 done71worker2 # Writing superblocks and filesystem accounting information: 0/8 done72client # 73worker2 # 74worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether75lb # Writing inode tables: 0/8 done76client # Virtualisation disk image created.77worker2 # Virtualisation disk image created.78worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...79lb # Creating journal (8192 blocks): done80client # Starting virtiofs daemons...81worker2 # Starting virtiofs daemons...82worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83lb # Writing superblocks and filesystem accounting information: 0/8 done84client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87lb # 88client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...91lb # Virtualisation disk image created.92client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)93worker2 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...94worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95lb # Starting virtiofs daemons...96client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether97worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)98worker1 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether99lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100client # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...101worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether102worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...103lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)105worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)106worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Client connected, servicing requests107lb # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...108client # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether109worker2 # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether110worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Client connected, servicing requests111lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)112client # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...113worker2 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...114worker1 # [2026-09-23T05:31:58Z INFO virtiofsd] Client connected, servicing requests115lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether116client # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...117lb # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...118client # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests119lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests121lb # [2026-09-23T05:31:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether122client # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests123lb # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...124worker2 # [2026-09-23T05:31:58Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests126worker2 # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests127lb # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests128worker2 # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests130worker2 # [2026-09-23T05:31:59Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker1 # [ 0.000000] KASLR enabled134worker1 # [ 0.000000] random: crng init done135worker1 # [ 0.000000] Machine model: linux,dummy-virt136worker1 # [ 0.000000] efi: UEFI not found.137worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140worker1 # [ 0.000000] Zone ranges:141worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142worker1 # [ 0.000000] DMA32 empty143worker1 # [ 0.000000] Normal empty144worker1 # [ 0.000000] Device empty145worker1 # [ 0.000000] Movable zone start for each node146worker1 # [ 0.000000] Early memory node ranges147worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker1 # [ 0.000000] psci: probing for conduit method from DT.151worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153worker1 # [ 0.000000] psci: Trusted OS migration not required154worker1 # [ 0.000000] psci: SMC Calling Convention v1.1155worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157worker1 # [ 0.000000] Detected PIPT I-cache on CPU0158worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface160worker1 # [ 0.000000] CPU features: detected: Spectre-v4161worker1 # [ 0.000000] CPU features: detected: Spectre-BHB162worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164worker1 # [ 0.000000] alternatives: applying boot alternatives165worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4cjy3h5garv6kdzqg4fsdb44fsjfs9vp-nixos-system-worker1-test/init regInfo=/nix/store/kilsa45kilzg2azh7zrbnmf6qgdvl019-closure-info/registration console=ttyAMA0,115200n8 console=tty0166worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kilsa45kilzg2azh7zrbnmf6qgdvl019-closure-info/registration", will be passed to user space.167worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)169worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)170worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB171worker1 # [ 0.000000] software IO TLB: area num 1.172worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB173worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)174worker1 # [ 0.000000] Fallback order for Node 0: 0175worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216176worker1 # [ 0.000000] Policy zone: DMA177worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off178worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1179worker1 # [ 0.000000] allocated 3145728 bytes of page_ext180worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages181worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups182worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.183worker1 # [ 0.000000] rcu: RCU event tracing is enabled.184worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.185worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.186worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.187worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.188worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.189worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1190worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.191worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.192worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.193worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0194worker1 # [ 0.000000] GICv3: 256 SPIs implemented195worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented196worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq197worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI198worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0199worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000200worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]201worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)202worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)203worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000204worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000205worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.206worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns207worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).208worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns209worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns210worker1 # [ 0.000033] arm-pv: using stolen time PV211worker1 # [ 0.000645] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)212worker1 # [ 0.000802] Console: colour dummy device 80x25213worker1 # [ 0.000809] printk: legacy console [tty0] enabled214worker1 # [ 0.001022] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)215worker1 # [ 0.001029] pid_max: default: 32768 minimum: 301216worker1 # [ 0.001107] LSM: initializing lsm=capability,landlock,yama,bpf,ima217worker1 # [ 0.001280] landlock: Up and running.218worker1 # [ 0.001283] Yama: becoming mindful.219worker1 # [ 0.001872] LSM support for eBPF active220worker1 # [ 0.002026] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)221worker1 # [ 0.002059] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)222worker1 # [ 0.003931] rcu: Hierarchical SRCU implementation.223worker1 # [ 0.003935] rcu: Max phase no-delay instances is 1000.224worker1 # [ 0.005168] fsl-mc MSI: its@8080000 domain created225worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]226worker1 # [ 0.005257] EFI services will not be available.227worker1 # [ 0.005334] smp: Bringing up secondary CPUs ...228worker1 # [ 0.005342] smp: Brought up 1 node, 1 CPU229worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026230worker2 # [ 0.000000] KASLR enabled231worker1 # [ 0.005345] SMP: Total of 1 processors activated.232worker2 # [ 0.000000] random: crng init done233worker1 # [ 0.005350] CPU: All CPU(s) started at EL1234worker2 # [ 0.000000] Machine model: linux,dummy-virt235worker1 # [ 0.005360] CPU features: detected: Branch Target Identification236worker2 # [ 0.000000] efi: UEFI not found.237worker1 # [ 0.005365] CPU features: detected: ARMv8.4 Translation Table Level238worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT239worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]240worker1 # [ 0.005368] CPU features: detected: Instruction cache invalidation not required for I/D coherence241lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]242worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]243worker2 # [ 0.000000] Zone ranges:244worker1 # [ 0.005372] CPU features: detected: Data cache clean to the PoU not required for I/D coherence245lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026246worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]247worker1 # [ 0.005376] CPU features: detected: Common not Private translations248lb # [ 0.000000] KASLR enabled249worker2 # [ 0.000000] DMA32 empty250worker1 # [ 0.005379] CPU features: detected: CRC32 instructions251lb # [ 0.000000] random: crng init done252worker2 # [ 0.000000] Normal empty253worker2 # [ 0.000000] Device empty254lb # [ 0.000000] Machine model: linux,dummy-virt255client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]256worker1 # [ 0.005382] CPU features: detected: Data cache clean to Point of Deep Persistence257lb # [ 0.000000] efi: UEFI not found.258worker2 # [ 0.000000] Movable zone start for each node259worker2 # [ 0.000000] Early memory node ranges260worker1 # [ 0.005385] CPU features: detected: Data cache clean to Point of Persistence261lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT262client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026263worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]264client # [ 0.000000] KASLR enabled265worker1 # [ 0.005388] CPU features: detected: Data independent timing control (DIT)266lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]267client # [ 0.000000] random: crng init done268worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]269worker1 # [ 0.005391] CPU features: detected: E0PD270client # [ 0.000000] Machine model: linux,dummy-virt271lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]272worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000273client # [ 0.000000] efi: UEFI not found.274lb # [ 0.000000] Zone ranges:275worker1 # [ 0.005394] CPU features: detected: Enhanced Counter Virtualization276worker2 # [ 0.000000] psci: probing for conduit method from DT.277lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]278client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT279worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.280lb # [ 0.000000] DMA32 empty281worker1 # [ 0.005397] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)282lb # [ 0.000000] Normal empty283worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs284client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]285worker1 # [ 0.005400] CPU features: detected: Enhanced Virtualization Traps286lb # [ 0.000000] Device empty287worker2 # [ 0.000000] psci: Trusted OS migration not required288lb # [ 0.000000] Movable zone start for each node289worker1 # [ 0.005403] CPU features: detected: Fine Grained Traps290client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]291client # [ 0.000000] Zone ranges:292worker2 # [ 0.000000] psci: SMC Calling Convention v1.1293lb # [ 0.000000] Early memory node ranges294worker1 # [ 0.005406] CPU features: detected: Generic authentication (architected QARMA5 algorithm)295client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]296lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]297client # [ 0.000000] DMA32 empty298worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)299worker1 # [ 0.005411] CPU features: detected: RCpc load-acquire (LDAPR)300client # [ 0.000000] Normal empty301lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]302client # [ 0.000000] Device empty303worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296304worker1 # [ 0.005414] CPU features: detected: LSE atomic instructions305lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000306client # [ 0.000000] Movable zone start for each node307worker2 # [ 0.000000] Detected PIPT I-cache on CPU0308worker1 # [ 0.005417] CPU features: detected: Privileged Access Never309client # [ 0.000000] Early memory node ranges310lb # [ 0.000000] psci: probing for conduit method from DT.311worker1 # [ 0.005419] CPU features: detected: PMUv3312worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)313lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.314client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]315worker1 # [ 0.005422] CPU features: detected: RAS Extension Support316worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface317lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs318client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]319worker1 # [ 0.005425] CPU features: detected: RASv1p1 Extension Support320worker2 # [ 0.000000] CPU features: detected: Spectre-v4321lb # [ 0.000000] psci: Trusted OS migration not required322client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000323worker2 # [ 0.000000] CPU features: detected: Spectre-BHB324worker1 # [ 0.005428] CPU features: detected: Random Number Generator325lb # [ 0.000000] psci: SMC Calling Convention v1.1326client # [ 0.000000] psci: probing for conduit method from DT.327worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38328client # [ 0.000000] psci: PSCIv1.3 detected in firmware.329worker1 # [ 0.005430] CPU features: detected: Speculation barrier (SB)330lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)331worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23332client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs333worker1 # [ 0.005433] CPU features: detected: Stage-2 Force Write-Back334lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296335worker2 # [ 0.000000] alternatives: applying boot alternatives336client # [ 0.000000] psci: Trusted OS migration not required337worker1 # [ 0.005436] CPU features: detected: TLB range maintenance instructions338lb # [ 0.000000] Detected PIPT I-cache on CPU0339client # [ 0.000000] psci: SMC Calling Convention v1.1340worker1 # [ 0.005441] CPU features: detected: Speculative Store Bypassing Safe (SSBS)341lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)342client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)343worker1 # [ 0.005480] alternatives: applying system-wide alternatives344lb # [ 0.000000] CPU features: detected: GICv3 CPU interface345client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296346worker1 # [ 0.008358] CPU features: detected: BBM Level 2 without TLB conflict abort347lb # [ 0.000000] CPU features: detected: Spectre-v4348client # [ 0.000000] Detected PIPT I-cache on CPU0349lb # [ 0.000000] CPU features: detected: Spectre-BHB350lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38351client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)352worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bszflazskwamnpdm0gzh5fsvsms7kvza-nixos-system-worker2-test/init regInfo=/nix/store/0fpyr3903f58d5irnb20j9w5pnq32y4y-closure-info/registration console=ttyAMA0,115200n8 console=tty0353worker1 # [ 0.008519] Memory: 1405640K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)354lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23355client # [ 0.000000] CPU features: detected: GICv3 CPU interface356worker1 # [ 0.008889] devtmpfs: initialized357client # [ 0.000000] CPU features: detected: Spectre-v4358lb # [ 0.000000] alternatives: applying boot alternatives359worker1 # [ 0.010640] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)360worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0fpyr3903f58d5irnb20j9w5pnq32y4y-closure-info/registration", will be passed to user space.361client # [ 0.000000] CPU features: detected: Spectre-BHB362client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38363worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes364worker1 # [ 0.010662] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).365client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23366worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)367worker1 # [ 0.010840] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL368client # [ 0.000000] alternatives: applying boot alternatives369worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)370worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB371lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3in2qynk4v72bmzzvgahb746q83gs855-nixos-system-lb-test/init regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration console=ttyAMA0,115200n8 console=tty0372worker2 # [ 0.000000] software IO TLB: area num 1.373worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB374lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration", will be passed to user space.375worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)376client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test/init regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration console=ttyAMA0,115200n8 console=tty0377lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes378worker2 # [ 0.000000] Fallback order for Node 0: 0379worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216380lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)381worker2 # [ 0.000000] Policy zone: DMA382client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration", will be passed to user space.383lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)384worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off385client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes386worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1387worker2 # [ 0.000000] allocated 3145728 bytes of page_ext388lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB389client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)390lb # [ 0.000000] software IO TLB: area num 1.391worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages392client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)393worker1 # [ 0.010844] 0 pages in range for non-PLT usage394worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups395lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB396worker1 # [ 0.010845] 508272 pages in range for PLT usage397client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB398worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.399lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)400client # [ 0.000000] software IO TLB: area num 1.401worker1 # [ 0.010968] pinctrl core: initialized pinctrl subsystem402worker2 # [ 0.000000] rcu: RCU event tracing is enabled.403lb # [ 0.000000] Fallback order for Node 0: 0404worker1 # [ 0.011739] DMI not present or invalid.405client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB406worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.407worker1 # [ 0.015398] NET: Registered PF_NETLINK/PF_ROUTE protocol family408lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216409worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.410lb # [ 0.000000] Policy zone: DMA411client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)412worker1 # [ 0.017551] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations413worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.414client # [ 0.000000] Fallback order for Node 0: 0415lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off416worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.417client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216418worker1 # [ 0.017728] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations419lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1420client # [ 0.000000] Policy zone: DMA421worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.422lb # [ 0.000000] allocated 3145728 bytes of page_ext423worker1 # [ 0.017926] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations424client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off425worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1426lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages427worker1 # [ 0.017947] audit: initializing netlink subsys (disabled)428client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1429lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups430worker1 # [ 0.018504] thermal_sys: Registered thermal governor 'fair_share'431client # [ 0.000000] allocated 3145728 bytes of page_ext432worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.433lb # [ 0.000000] rcu: Hierarchical RCU implementation.434client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages435worker1 # [ 0.018506] thermal_sys: Registered thermal governor 'bang_bang'436lb # [ 0.000000] rcu: RCU event tracing is enabled.437worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.438client # [ 0.000000] ftrace: allocated 294 pages with 4 groups439worker1 # [ 0.018509] thermal_sys: Registered thermal governor 'step_wise'440lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.441client # [ 0.000000] rcu: Hierarchical RCU implementation.442worker1 # [ 0.018512] thermal_sys: Registered thermal governor 'user_space'443worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.444lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.445client # [ 0.000000] rcu: RCU event tracing is enabled.446lb # [ 0.000000] Rude variant of Tasks RCU enabled.447worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0448worker1 # [ 0.018517] thermal_sys: Registered thermal governor 'power_allocator'449client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.450worker2 # [ 0.000000] GICv3: 256 SPIs implemented451lb # [ 0.000000] Tracing variant of Tasks RCU enabled.452worker1 # [ 0.018543] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1453worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented454client # [ 0.000000] Trampoline variant of Tasks RCU enabled.455lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.456worker1 # [ 0.018552] cpuidle: using governor ladder457client # [ 0.000000] Rude variant of Tasks RCU enabled.458worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq459worker1 # [ 0.018558] cpuidle: using governor menu460client # [ 0.000000] Tracing variant of Tasks RCU enabled.461worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI462lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1463worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0464worker1 # [ 0.018753] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.465client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.466lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.467worker1 # [ 0.018768] ASID allocator initialised with 65536 entries468worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000469client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1470worker1 # [ 0.020021] Serial: AMBA PL011 UART driver471worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]472lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.473client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.474worker1 # [ 0.025384] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1475worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)476lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.477worker1 # [ 0.025534] printk: console [ttyAMA0] enabled478client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.479lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0480worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)481lb # [ 0.000000] GICv3: 256 SPIs implemented482worker1 # [ 0.147082] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages483client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.484worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000485lb # [ 0.000000] GICv3: 0 Extended SPIs implemented486worker1 # [ 0.147098] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page487client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0488lb # [ 0.000000] Root IRQ handler: gic_handle_irq489worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000490client # [ 0.000000] GICv3: 256 SPIs implemented491lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI492worker1 # [ 0.147103] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages493client # [ 0.000000] GICv3: 0 Extended SPIs implemented494worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.495lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0496client # [ 0.000000] Root IRQ handler: gic_handle_irq497worker1 # [ 0.147107] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page498lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000499client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI500worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns501worker1 # [ 0.147111] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages502lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]503client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0504worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).505worker1 # [ 0.147116] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page506client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000507lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)508worker1 # [ 0.147120] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages509client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]510worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns511worker1 # [ 0.147124] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page512lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)513client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)514worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns515lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000516worker2 # [ 0.000029] arm-pv: using stolen time PV517client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)518lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000519worker2 # [ 0.000385] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)520client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000521worker1 # [ 0.154225] fbcon: Taking over console522lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.523worker2 # [ 0.000558] Console: colour dummy device 80x25524worker1 # [ 0.154236] ACPI: Interpreter disabled.525client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000526worker2 # [ 0.000566] printk: legacy console [tty0] enabled527worker1 # [ 0.156078] iommu: Default domain type: Translated528lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns529client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.530worker1 # [ 0.156087] iommu: DMA domain TLB invalidation policy: strict mode531lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).532worker2 # [ 0.000761] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)533worker2 # [ 0.000767] pid_max: default: 32768 minimum: 301534client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns535worker1 # [ 0.157695] SCSI subsystem initialized536client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).537worker2 # [ 0.000838] LSM: initializing lsm=capability,landlock,yama,bpf,ima538lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns539worker2 # [ 0.000975] landlock: Up and running.540worker2 # [ 0.000978] Yama: becoming mindful.541lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns542client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns543worker2 # [ 0.001414] LSM support for eBPF active544lb # [ 0.000027] arm-pv: using stolen time PV545client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns546worker2 # [ 0.001547] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)547lb # [ 0.000400] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)548client # [ 0.000028] arm-pv: using stolen time PV549worker2 # [ 0.001579] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)550lb # [ 0.000595] Console: colour dummy device 80x25551worker2 # [ 0.003300] rcu: Hierarchical SRCU implementation.552lb # [ 0.000602] printk: legacy console [tty0] enabled553client # [ 0.000361] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)554worker2 # [ 0.003305] rcu: Max phase no-delay instances is 1000.555client # [ 0.000524] Console: colour dummy device 80x25556worker2 # [ 0.004518] fsl-mc MSI: its@8080000 domain created557client # [ 0.000531] printk: legacy console [tty0] enabled558lb # [ 0.000786] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)559worker2 # [ 0.004607] EFI services will not be available.560lb # [ 0.000793] pid_max: default: 32768 minimum: 301561worker2 # [ 0.004670] smp: Bringing up secondary CPUs ...562client # [ 0.000723] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)563lb # [ 0.000888] LSM: initializing lsm=capability,landlock,yama,bpf,ima564worker2 # [ 0.004678] smp: Brought up 1 node, 1 CPU565lb # [ 0.001006] landlock: Up and running.566client # [ 0.000729] pid_max: default: 32768 minimum: 301567worker2 # [ 0.004681] SMP: Total of 1 processors activated.568lb # [ 0.001008] Yama: becoming mindful.569client # [ 0.000810] LSM: initializing lsm=capability,landlock,yama,bpf,ima570worker2 # [ 0.004686] CPU: All CPU(s) started at EL1571lb # [ 0.001446] LSM support for eBPF active572worker1 # [ 0.165264] usbcore: registered new interface driver usbfs573client # [ 0.000934] landlock: Up and running.574worker2 # [ 0.004696] CPU features: detected: Branch Target Identification575client # [ 0.000937] Yama: becoming mindful.576worker1 # [ 0.165297] usbcore: registered new interface driver hub577lb # [ 0.001601] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)578client # [ 0.001344] LSM support for eBPF active579worker2 # [ 0.004699] CPU features: detected: ARMv8.4 Translation Table Level580worker1 # [ 0.165313] usbcore: registered new device driver usb581lb # [ 0.001633] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)582worker1 # [ 0.165577] pps_core: LinuxPPS API ver. 1 registered583client # [ 0.001480] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)584lb # [ 0.003412] rcu: Hierarchical SRCU implementation.585worker2 # [ 0.004702] CPU features: detected: Instruction cache invalidation not required for I/D coherence586client # [ 0.001513] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)587lb # [ 0.003416] rcu: Max phase no-delay instances is 1000.588worker1 # [ 0.165583] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>589worker1 # [ 0.165593] PTP clock support registered590worker2 # [ 0.004706] CPU features: detected: Data cache clean to the PoU not required for I/D coherence591client # [ 0.003315] rcu: Hierarchical SRCU implementation.592lb # [ 0.004624] fsl-mc MSI: its@8080000 domain created593worker1 # [ 0.165659] EDAC MC: Ver: 3.0.0594client # [ 0.003319] rcu: Max phase no-delay instances is 1000.595lb # [ 0.004720] EFI services will not be available.596worker2 # [ 0.004710] CPU features: detected: Common not Private translations597client # [ 0.004590] fsl-mc MSI: its@8080000 domain created598lb # [ 0.004795] smp: Bringing up secondary CPUs ...599worker2 # [ 0.004713] CPU features: detected: CRC32 instructions600worker1 # [ 0.170086] scmi_core: SCMI protocol bus registered601lb # [ 0.004803] smp: Brought up 1 node, 1 CPU602client # [ 0.004678] EFI services will not be available.603worker2 # [ 0.004716] CPU features: detected: Data cache clean to Point of Deep Persistence604lb # [ 0.004806] SMP: Total of 1 processors activated.605client # [ 0.004748] smp: Bringing up secondary CPUs ...606worker1 # [ 0.171027] FPGA manager framework607lb # [ 0.004811] CPU: All CPU(s) started at EL1608client # [ 0.004756] smp: Brought up 1 node, 1 CPU609worker1 # [ 0.171974] vgaarb: loaded610worker2 # [ 0.004720] CPU features: detected: Data cache clean to Point of Persistence611client # [ 0.004760] SMP: Total of 1 processors activated.612lb # [ 0.004821] CPU features: detected: Branch Target Identification613client # [ 0.004765] CPU: All CPU(s) started at EL1614worker2 # [ 0.004723] CPU features: detected: Data independent timing control (DIT)615lb # [ 0.004824] CPU features: detected: ARMv8.4 Translation Table Level616worker2 # [ 0.004726] CPU features: detected: E0PD617worker1 # [ 0.172564] clocksource: Switched to clocksource arch_sys_counter618client # [ 0.004776] CPU features: detected: Branch Target Identification619worker1 # [ 0.173126] VFS: Disk quotas dquot_6.6.0620worker2 # [ 0.004728] CPU features: detected: Enhanced Counter Virtualization621client # [ 0.004781] CPU features: detected: ARMv8.4 Translation Table Level622lb # [ 0.004828] CPU features: detected: Instruction cache invalidation not required for I/D coherence623worker1 # [ 0.173153] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)624worker2 # [ 0.004731] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)625lb # [ 0.004831] CPU features: detected: Data cache clean to the PoU not required for I/D coherence626client # [ 0.004784] CPU features: detected: Instruction cache invalidation not required for I/D coherence627worker2 # [ 0.004735] CPU features: detected: Enhanced Virtualization Traps628worker1 # [ 0.175394] netfs: FS-Cache loaded629worker1 # [ 0.175502] pnp: PnP ACPI: disabled630lb # [ 0.004835] CPU features: detected: Common not Private translations631worker2 # [ 0.004738] CPU features: detected: Fine Grained Traps632client # [ 0.004788] CPU features: detected: Data cache clean to the PoU not required for I/D coherence633lb # [ 0.004839] CPU features: detected: CRC32 instructions634client # [ 0.004792] CPU features: detected: Common not Private translations635worker2 # [ 0.004742] CPU features: detected: Generic authentication (architected QARMA5 algorithm)636lb # [ 0.004842] CPU features: detected: Data cache clean to Point of Deep Persistence637client # [ 0.004795] CPU features: detected: CRC32 instructions638worker2 # [ 0.004747] CPU features: detected: RCpc load-acquire (LDAPR)639lb # [ 0.004845] CPU features: detected: Data cache clean to Point of Persistence640worker2 # [ 0.004750] CPU features: detected: LSE atomic instructions641client # [ 0.004798] CPU features: detected: Data cache clean to Point of Deep Persistence642worker2 # [ 0.004753] CPU features: detected: Privileged Access Never643lb # [ 0.004848] CPU features: detected: Data independent timing control (DIT)644client # [ 0.004801] CPU features: detected: Data cache clean to Point of Persistence645worker2 # [ 0.004756] CPU features: detected: PMUv3646lb # [ 0.004852] CPU features: detected: E0PD647client # [ 0.004804] CPU features: detected: Data independent timing control (DIT)648worker2 # [ 0.004759] CPU features: detected: RAS Extension Support649lb # [ 0.004854] CPU features: detected: Enhanced Counter Virtualization650client # [ 0.004807] CPU features: detected: E0PD651worker2 # [ 0.004761] CPU features: detected: RASv1p1 Extension Support652lb # [ 0.004857] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)653client # [ 0.004810] CPU features: detected: Enhanced Counter Virtualization654worker2 # [ 0.004764] CPU features: detected: Random Number Generator655lb # [ 0.004861] CPU features: detected: Enhanced Virtualization Traps656client # [ 0.004813] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)657worker2 # [ 0.004767] CPU features: detected: Speculation barrier (SB)658lb # [ 0.004864] CPU features: detected: Fine Grained Traps659client # [ 0.004816] CPU features: detected: Enhanced Virtualization Traps660worker2 # [ 0.004769] CPU features: detected: Stage-2 Force Write-Back661client # [ 0.004820] CPU features: detected: Fine Grained Traps662lb # [ 0.004867] CPU features: detected: Generic authentication (architected QARMA5 algorithm)663worker2 # [ 0.004772] CPU features: detected: TLB range maintenance instructions664worker2 # [ 0.004777] CPU features: detected: Speculative Store Bypassing Safe (SSBS)665client # [ 0.004823] CPU features: detected: Generic authentication (architected QARMA5 algorithm)666worker1 # [ 0.181411] NET: Registered PF_INET protocol family667worker2 # [ 0.004814] alternatives: applying system-wide alternatives668client # [ 0.004828] CPU features: detected: RCpc load-acquire (LDAPR)669client # [ 0.004831] CPU features: detected: LSE atomic instructions670worker1 # [ 0.181667] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)671worker2 # [ 0.007660] CPU features: detected: BBM Level 2 without TLB conflict abort672client # [ 0.004834] CPU features: detected: Privileged Access Never673client # [ 0.004836] CPU features: detected: PMUv3674worker2 # [ 0.007867] Memory: 1405664K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)675client # [ 0.004839] CPU features: detected: RAS Extension Support676worker2 # [ 0.008264] devtmpfs: initialized677client # [ 0.004842] CPU features: detected: RASv1p1 Extension Support678worker2 # [ 0.010022] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)679client # [ 0.004844] CPU features: detected: Random Number Generator680client # [ 0.004847] CPU features: detected: Speculation barrier (SB)681worker2 # [ 0.010043] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).682client # [ 0.004850] CPU features: detected: Stage-2 Force Write-Back683worker2 # [ 0.010212] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL684worker2 # [ 0.010216] 0 pages in range for non-PLT usage685client # [ 0.004852] CPU features: detected: TLB range maintenance instructions686worker2 # [ 0.010217] 508272 pages in range for PLT usage687client # [ 0.004857] CPU features: detected: Speculative Store Bypassing Safe (SSBS)688lb # [ 0.004872] CPU features: detected: RCpc load-acquire (LDAPR)689worker2 # [ 0.010304] pinctrl core: initialized pinctrl subsystem690client # [ 0.004891] alternatives: applying system-wide alternatives691worker2 # [ 0.011012] DMI not present or invalid.692lb # [ 0.004875] CPU features: detected: LSE atomic instructions693client # [ 0.007762] CPU features: detected: BBM Level 2 without TLB conflict abort694worker2 # [ 0.013975] NET: Registered PF_NETLINK/PF_ROUTE protocol family695lb # [ 0.004878] CPU features: detected: Privileged Access Never696lb # [ 0.004881] CPU features: detected: PMUv3697worker2 # [ 0.016348] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations698lb # [ 0.004884] CPU features: detected: RAS Extension Support699client # [ 0.007961] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)700worker2 # [ 0.016535] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations701lb # [ 0.004887] CPU features: detected: RASv1p1 Extension Support702client # [ 0.008385] devtmpfs: initialized703lb # [ 0.004889] CPU features: detected: Random Number Generator704worker2 # [ 0.016728] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations705client # [ 0.010159] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)706worker2 # [ 0.016747] audit: initializing netlink subsys (disabled)707lb # [ 0.004892] CPU features: detected: Speculation barrier (SB)708client # [ 0.010181] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).709lb # [ 0.004894] CPU features: detected: Stage-2 Force Write-Back710worker2 # [ 0.017283] thermal_sys: Registered thermal governor 'fair_share'711client # [ 0.010349] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL712lb # [ 0.004897] CPU features: detected: TLB range maintenance instructions713worker2 # [ 0.017285] thermal_sys: Registered thermal governor 'bang_bang'714client # [ 0.010353] 0 pages in range for non-PLT usage715worker2 # [ 0.017288] thermal_sys: Registered thermal governor 'step_wise'716lb # [ 0.004902] CPU features: detected: Speculative Store Bypassing Safe (SSBS)717client # [ 0.010354] 508272 pages in range for PLT usage718worker2 # [ 0.017291] thermal_sys: Registered thermal governor 'user_space'719lb # [ 0.004935] alternatives: applying system-wide alternatives720client # [ 0.010448] pinctrl core: initialized pinctrl subsystem721client # [ 0.011204] DMI not present or invalid.722worker2 # [ 0.017296] thermal_sys: Registered thermal governor 'power_allocator'723lb # [ 0.007824] CPU features: detected: BBM Level 2 without TLB conflict abort724client # [ 0.014291] NET: Registered PF_NETLINK/PF_ROUTE protocol family725worker2 # [ 0.017325] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1726worker2 # [ 0.017335] cpuidle: using governor ladder727client # [ 0.016432] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations728lb # [ 0.008086] Memory: 1405696K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)729worker2 # [ 0.017341] cpuidle: using governor menu730lb # [ 0.008457] devtmpfs: initialized731client # [ 0.016624] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations732worker2 # [ 0.017515] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.733lb # [ 0.010263] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)734client # [ 0.016821] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations735worker2 # [ 0.017530] ASID allocator initialised with 65536 entries736worker2 # [ 0.018723] Serial: AMBA PL011 UART driver737client # [ 0.016842] audit: initializing netlink subsys (disabled)738lb # [ 0.010290] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).739client # [ 0.017360] thermal_sys: Registered thermal governor 'fair_share'740lb # [ 0.010463] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL741client # [ 0.017362] thermal_sys: Registered thermal governor 'bang_bang'742lb # [ 0.010467] 0 pages in range for non-PLT usage743worker2 # [ 0.023949] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1744lb # [ 0.010469] 508272 pages in range for PLT usage745worker2 # [ 0.024120] printk: console [ttyAMA0] enabled746client # [ 0.017366] thermal_sys: Registered thermal governor 'step_wise'747lb # [ 0.010576] pinctrl core: initialized pinctrl subsystem748client # [ 0.017368] thermal_sys: Registered thermal governor 'user_space'749lb # [ 0.011355] DMI not present or invalid.750client # [ 0.017373] thermal_sys: Registered thermal governor 'power_allocator'751lb # [ 0.014466] NET: Registered PF_NETLINK/PF_ROUTE protocol family752client # [ 0.017396] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1753lb # [ 0.016665] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations754client # [ 0.017407] cpuidle: using governor ladder755client # [ 0.017412] cpuidle: using governor menu756lb # [ 0.016838] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations757client # [ 0.017596] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.758lb # [ 0.017041] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations759client # [ 0.017610] ASID allocator initialised with 65536 entries760lb # [ 0.017062] audit: initializing netlink subsys (disabled)761client # [ 0.018781] Serial: AMBA PL011 UART driver762lb # [ 0.017586] thermal_sys: Registered thermal governor 'fair_share'763client # [ 0.023908] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1764lb # [ 0.017588] thermal_sys: Registered thermal governor 'bang_bang'765client # [ 0.024039] printk: console [ttyAMA0] enabled766worker2 # [ 0.140753] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages767lb # [ 0.017592] thermal_sys: Registered thermal governor 'step_wise'768worker2 # [ 0.140767] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page769lb # [ 0.017594] thermal_sys: Registered thermal governor 'user_space'770worker2 # [ 0.140773] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages771lb # [ 0.017600] thermal_sys: Registered thermal governor 'power_allocator'772worker2 # [ 0.140777] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page773lb # [ 0.017622] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1774worker2 # [ 0.140782] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages775lb # [ 0.017632] cpuidle: using governor ladder776lb # [ 0.017638] cpuidle: using governor menu777worker2 # [ 0.140786] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page778worker1 # [ 0.209860] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)779lb # [ 0.017820] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.780worker2 # [ 0.140790] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages781worker1 # [ 0.209895] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)782lb # [ 0.017833] ASID allocator initialised with 65536 entries783worker2 # [ 0.140794] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page784lb # [ 0.019038] Serial: AMBA PL011 UART driver785worker1 # [ 0.209919] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)786lb # [ 0.024307] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1787worker1 # [ 0.209988] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)788lb # [ 0.024452] printk: console [ttyAMA0] enabled789client # [ 0.140151] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages790worker1 # [ 0.210131] TCP: Hash tables configured (established 16384 bind 16384)791worker2 # [ 0.148110] fbcon: Taking over console792client # [ 0.140166] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page793worker2 # [ 0.148123] ACPI: Interpreter disabled.794worker1 # [ 0.210201] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)795lb # [ 0.144565] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages796client # [ 0.140172] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages797worker1 # [ 0.210233] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)798lb # [ 0.144583] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page799client # [ 0.140176] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page800worker1 # [ 0.210268] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)801lb # [ 0.144589] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages802client # [ 0.140181] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages803worker1 # [ 0.210364] NET: Registered PF_UNIX/PF_LOCAL protocol family804lb # [ 0.144593] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page805client # [ 0.140185] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page806worker1 # [ 0.210383] NET: Registered PF_XDP protocol family807lb # [ 0.144597] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages808client # [ 0.140190] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages809worker1 # [ 0.210398] PCI: CLS 0 bytes, default 64810lb # [ 0.144602] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page811worker1 # [ 0.210610] Trying to unpack rootfs image as initramfs...812client # [ 0.140194] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page813lb # [ 0.144606] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages814lb # [ 0.144611] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page815client # [ 0.147345] fbcon: Taking over console816client # [ 0.147358] ACPI: Interpreter disabled.817lb # [ 0.151885] fbcon: Taking over console818worker2 # [ 0.156966] iommu: Default domain type: Translated819lb # [ 0.151897] ACPI: Interpreter disabled.820worker2 # [ 0.156976] iommu: DMA domain TLB invalidation policy: strict mode821worker2 # [ 0.157357] SCSI subsystem initialized822worker2 # [ 0.159334] usbcore: registered new interface driver usbfs823worker2 # [ 0.159365] usbcore: registered new interface driver hub824worker1 # [ 0.226025] kvm [1]: HYP mode not available825worker2 # [ 0.159380] usbcore: registered new device driver usb826worker2 # [ 0.159652] pps_core: LinuxPPS API ver. 1 registered827worker2 # [ 0.159658] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>828worker2 # [ 0.159668] PTP clock support registered829worker2 # [ 0.159722] EDAC MC: Ver: 3.0.0830lb # [ 0.158918] iommu: Default domain type: Translated831lb # [ 0.158926] iommu: DMA domain TLB invalidation policy: strict mode832client # [ 0.156340] iommu: Default domain type: Translated833worker2 # [ 0.164294] scmi_core: SCMI protocol bus registered834lb # [ 0.160567] SCSI subsystem initialized835client # [ 0.156349] iommu: DMA domain TLB invalidation policy: strict mode836worker2 # [ 0.165237] FPGA manager framework837client # [ 0.156677] SCSI subsystem initialized838worker2 # [ 0.166207] vgaarb: loaded839lb # [ 0.161238] usbcore: registered new interface driver usbfs840worker2 # [ 0.166822] clocksource: Switched to clocksource arch_sys_counter841client # [ 0.158642] usbcore: registered new interface driver usbfs842lb # [ 0.161270] usbcore: registered new interface driver hub843client # [ 0.158673] usbcore: registered new interface driver hub844lb # [ 0.161286] usbcore: registered new device driver usb845client # [ 0.158688] usbcore: registered new device driver usb846lb # [ 0.161561] pps_core: LinuxPPS API ver. 1 registered847client # [ 0.158960] pps_core: LinuxPPS API ver. 1 registered848lb # [ 0.161567] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>849lb # [ 0.161578] PTP clock support registered850client # [ 0.158966] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>851lb # [ 0.161630] EDAC MC: Ver: 3.0.0852client # [ 0.158976] PTP clock support registered853worker2 # [ 0.170434] VFS: Disk quotas dquot_6.6.0854client # [ 0.159033] EDAC MC: Ver: 3.0.0855lb # [ 0.166272] scmi_core: SCMI protocol bus registered856worker2 # [ 0.170468] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)857client # [ 0.163649] scmi_core: SCMI protocol bus registered858lb # [ 0.167236] FPGA manager framework859lb # [ 0.168230] vgaarb: loaded860client # [ 0.164610] FPGA manager framework861client # [ 0.165543] vgaarb: loaded862lb # [ 0.168842] clocksource: Switched to clocksource arch_sys_counter863client # [ 0.166167] clocksource: Switched to clocksource arch_sys_counter864worker2 # [ 0.174200] netfs: FS-Cache loaded865worker2 # [ 0.174315] pnp: PnP ACPI: disabled866lb # [ 0.172121] VFS: Disk quotas dquot_6.6.0867worker2 # [ 0.178205] NET: Registered PF_INET protocol family868lb # [ 0.172156] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)869worker2 # [ 0.178463] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)870client # [ 0.172327] VFS: Disk quotas dquot_6.6.0871lb # [ 0.175747] netfs: FS-Cache loaded872lb # [ 0.175878] pnp: PnP ACPI: disabled873client # [ 0.172371] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)874client # [ 0.174027] netfs: FS-Cache loaded875client # [ 0.174150] pnp: PnP ACPI: disabled876lb # [ 0.179640] NET: Registered PF_INET protocol family877client # [ 0.177915] NET: Registered PF_INET protocol family878lb # [ 0.179890] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)879client # [ 0.178170] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)880lb # [ 0.208388] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881lb # [ 0.208429] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882worker2 # [ 0.207136] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)883lb # [ 0.208454] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)884worker2 # [ 0.207176] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)885lb # [ 0.208523] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)886worker2 # [ 0.207201] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)887lb # [ 0.208666] TCP: Hash tables configured (established 16384 bind 16384)888worker2 # [ 0.207270] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)889lb # [ 0.208744] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)890worker2 # [ 0.207413] TCP: Hash tables configured (established 16384 bind 16384)891lb # [ 0.208777] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)892worker2 # [ 0.207482] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)893lb # [ 0.208812] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)894worker2 # [ 0.207514] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)895worker2 # [ 0.207549] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)896worker2 # [ 0.207648] NET: Registered PF_UNIX/PF_LOCAL protocol family897client # [ 0.206974] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)898worker2 # [ 0.207668] NET: Registered PF_XDP protocol family899worker2 # [ 0.207684] PCI: CLS 0 bytes, default 64900client # [ 0.207014] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)901worker2 # [ 0.207915] Trying to unpack rootfs image as initramfs...902client # [ 0.207038] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)903client # [ 0.207109] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)904client # [ 0.207251] TCP: Hash tables configured (established 16384 bind 16384)905client # [ 0.207334] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)906client # [ 0.207370] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)907client # [ 0.207405] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)908client # [ 0.207499] NET: Registered PF_UNIX/PF_LOCAL protocol family909lb # [ 0.216242] NET: Registered PF_UNIX/PF_LOCAL protocol family910client # [ 0.207524] NET: Registered PF_XDP protocol family911lb # [ 0.216266] NET: Registered PF_XDP protocol family912client # [ 0.207540] PCI: CLS 0 bytes, default 64913lb # [ 0.216283] PCI: CLS 0 bytes, default 64914worker2 # [ 0.224483] kvm [1]: HYP mode not available915client # [ 0.207777] Trying to unpack rootfs image as initramfs...916lb # [ 0.216512] Trying to unpack rootfs image as initramfs...917lb # [ 0.227297] kvm [1]: HYP mode not available918worker1 # [ 0.309016] Initialise system trusted keyrings919client # [ 0.223871] kvm [1]: HYP mode not available920worker1 # [ 0.309718] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker1 # [ 0.316847] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker1 # [ 0.317652] 9p: Installing v9fs 9p2000 file system support923worker1 # [ 0.337697] Key type asymmetric registered924worker1 # [ 0.337715] Asymmetric key parser 'x509' registered925worker1 # [ 0.337770] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)926worker1 # [ 0.339860] io scheduler mq-deadline registered927worker1 # [ 0.339870] io scheduler kyber registered928worker1 # [ 0.353001] pl061_gpio 9030000.pl061: PL061 GPIO chip registered929worker1 # [ 0.354349] ledtrig-cpu: registered to indicate activity on CPUs930worker1 # [ 0.354749] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:931worker1 # [ 0.354765] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000932worker1 # [ 0.354777] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000933worker1 # [ 0.354784] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000934worker1 # [ 0.354810] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits935worker1 # [ 0.354831] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]936worker1 # [ 0.354908] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00937worker1 # [ 0.354918] pci_bus 0000:00: root bus resource [bus 00-ff]938worker1 # [ 0.354923] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]939worker1 # [ 0.354928] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]940worker1 # [ 0.354933] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]941worker1 # [ 0.355017] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint942worker1 # [ 0.355442] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint943worker1 # [ 0.355636] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]944worker1 # [ 0.355653] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]945worker1 # [ 0.355682] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]946worker1 # [ 0.355697] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]947worker1 # [ 0.356141] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint948worker1 # [ 0.356319] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]949worker1 # [ 0.356335] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]950worker1 # [ 0.356363] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]951worker1 # [ 0.375823] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint952worker1 # [ 0.376005] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]953worker1 # [ 0.376021] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]954worker1 # [ 0.376050] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]955worker1 # [ 0.376493] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint956worker2 # [ 0.315395] Initialise system trusted keyrings957worker2 # [ 0.316145] workingset: timestamp_bits=42 max_order=19 bucket_order=0958worker2 # [ 0.317375] squashfs: version 4.0 (2009/01/31) Phillip Lougher959worker2 # [ 0.318160] 9p: Installing v9fs 9p2000 file system support960client # [ 0.314694] Initialise system trusted keyrings961client # [ 0.315444] workingset: timestamp_bits=42 max_order=19 bucket_order=0962client # [ 0.316752] squashfs: version 4.0 (2009/01/31) Phillip Lougher963worker1 # [ 0.388764] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]964worker1 # [ 0.388797] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]965client # [ 0.317533] 9p: Installing v9fs 9p2000 file system support966lb # [ 0.321412] Initialise system trusted keyrings967worker1 # [ 0.388827] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]968lb # [ 0.322154] workingset: timestamp_bits=42 max_order=19 bucket_order=0969worker1 # [ 0.388845] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]970lb # [ 0.323477] squashfs: version 4.0 (2009/01/31) Phillip Lougher971worker1 # [ 0.389338] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint972worker1 # [ 0.389528] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]973lb # [ 0.324252] 9p: Installing v9fs 9p2000 file system support974worker1 # [ 0.389558] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975worker1 # [ 0.390006] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint976worker1 # [ 0.390188] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]977worker1 # [ 0.390216] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]978worker1 # [ 0.390592] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint979worker1 # [ 0.390771] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]980worker1 # [ 0.391063] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint981worker1 # [ 0.391248] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]982worker1 # [ 0.391277] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983worker1 # [ 0.391731] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint984worker1 # [ 0.391912] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]985worker2 # [ 0.338813] Key type asymmetric registered986worker1 # [ 0.391941] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]987worker1 # [ 0.392378] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint988worker1 # [ 0.392559] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]989worker1 # [ 0.392600] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990worker1 # [ 0.393044] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint991worker1 # [ 0.393310] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]992worker1 # [ 0.393327] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]993worker1 # [ 0.393355] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker1 # [ 0.393819] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint995client # [ 0.338133] Key type asymmetric registered996worker1 # [ 0.393997] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]997client # [ 0.338156] Asymmetric key parser 'x509' registered998worker2 # [ 0.346871] Asymmetric key parser 'x509' registered999worker1 # [ 0.394012] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1000worker2 # [ 0.346956] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1001worker1 # [ 0.394041] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1002worker2 # [ 0.348607] io scheduler mq-deadline registered1003worker1 # [ 0.394612] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1004worker2 # [ 0.348619] io scheduler kyber registered1005worker1 # [ 0.394623] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1006worker1 # [ 0.394628] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1007worker1 # [ 0.394673] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1008worker1 # [ 0.394719] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1009lb # [ 0.348259] Key type asymmetric registered1010worker1 # [ 0.394766] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1011lb # [ 0.348284] Asymmetric key parser 'x509' registered1012client # [ 0.346252] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1013worker1 # [ 0.394813] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1014lb # [ 0.348348] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1015client # [ 0.347306] io scheduler mq-deadline registered1016worker1 # [ 0.394857] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1017client # [ 0.347317] io scheduler kyber registered1018worker1 # [ 0.394902] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1019worker1 # [ 0.394948] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1020worker1 # [ 0.394993] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1021worker1 # [ 0.395038] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1022worker2 # [ 0.358960] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1023worker1 # [ 0.395103] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1024worker1 # [ 0.395147] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1025worker1 # [ 0.395168] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1026worker1 # [ 0.395189] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1027worker2 # [ 0.360367] ledtrig-cpu: registered to indicate activity on CPUs1028worker1 # [ 0.395210] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1029worker2 # [ 0.360745] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1030worker1 # [ 0.395231] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1031worker2 # [ 0.360762] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001032worker1 # [ 0.395252] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1033lb # [ 0.356895] io scheduler mq-deadline registered1034lb # [ 0.356914] io scheduler kyber registered1035worker2 # [ 0.360783] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001036worker1 # [ 0.395273] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1037worker1 # [ 0.395297] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1038client # [ 0.358313] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1039worker2 # [ 0.360792] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001040lb # [ 0.362032] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1041worker1 # [ 0.395319] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1042worker2 # [ 0.360819] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1043worker1 # [ 0.395340] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1044worker2 # [ 0.360843] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1045worker1 # [ 0.395362] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1046lb # [ 0.363304] ledtrig-cpu: registered to indicate activity on CPUs1047worker2 # [ 0.360914] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001048client # [ 0.359697] ledtrig-cpu: registered to indicate activity on CPUs1049worker1 # [ 0.395383] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1050lb # [ 0.363679] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1051worker2 # [ 0.360923] pci_bus 0000:00: root bus resource [bus 00-ff]1052worker1 # [ 0.395404] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1053client # [ 0.360126] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1054worker2 # [ 0.360929] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1055worker1 # [ 0.395425] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1056lb # [ 0.363696] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001057client # [ 0.360143] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001058worker2 # [ 0.360935] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1059worker1 # [ 0.395445] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1060lb # [ 0.363708] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001061worker1 # [ 0.395466] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1062worker2 # [ 0.360940] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1063client # [ 0.360156] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001064lb # [ 0.363716] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001065worker1 # [ 0.395487] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1066worker2 # [ 0.361000] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1067client # [ 0.360165] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001068worker1 # [ 0.395508] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1069lb # [ 0.363736] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1070worker2 # [ 0.361439] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1071worker1 # [ 0.395532] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1072client # [ 0.360185] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1073worker2 # [ 0.361625] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1074lb # [ 0.363763] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1075worker1 # [ 0.395541] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1076client # [ 0.360212] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1077worker2 # [ 0.361641] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1078worker1 # [ 0.395546] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1079lb # [ 0.363832] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001080client # [ 0.360287] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001081worker2 # [ 0.361671] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1082lb # [ 0.363841] pci_bus 0000:00: root bus resource [bus 00-ff]1083worker1 # [ 0.396358] pci 0000:00:07.0: enabling device (0000 -> 0002)1084client # [ 0.360296] pci_bus 0000:00: root bus resource [bus 00-ff]1085worker2 # [ 0.361687] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1086lb # [ 0.363847] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1087client # [ 0.360302] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1088worker2 # [ 0.362153] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1089lb # [ 0.363852] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1090client # [ 0.360307] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1091worker2 # [ 0.362335] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1092lb # [ 0.363858] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1093client # [ 0.360312] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1094worker2 # [ 0.362351] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1095lb # [ 0.363910] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1096client # [ 0.360366] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1097worker2 # [ 0.362380] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098lb # [ 0.364345] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1099client # [ 0.360806] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1100lb # [ 0.364531] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1101client # [ 0.360994] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1102worker1 # [ 0.452694] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55019 usecs1103lb # [ 0.364547] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1104client # [ 0.361011] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1105worker2 # [ 0.381357] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1106lb # [ 0.364577] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1107worker2 # [ 0.381547] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1108client # [ 0.361041] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109lb # [ 0.364594] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1110worker2 # [ 0.381562] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1111client # [ 0.361057] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1112worker2 # [ 0.381592] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1113client # [ 0.361519] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1114client # [ 0.361705] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1115worker2 # [ 0.382052] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1116client # [ 0.361721] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1117worker2 # [ 0.382236] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1118worker2 # [ 0.382251] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1119client # [ 0.361750] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1120worker2 # [ 0.382281] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1121worker2 # [ 0.382299] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1122worker2 # [ 0.382762] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1123client # [ 0.380557] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1124client # [ 0.380744] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1125client # [ 0.380759] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1126client # [ 0.380789] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127client # [ 0.381247] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1128client # [ 0.381430] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1129client # [ 0.381446] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1130client # [ 0.381475] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1131client # [ 0.381494] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1132client # [ 0.381952] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1133lb # [ 0.385137] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1134client # [ 0.382139] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1135lb # [ 0.385339] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1136lb # [ 0.385357] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1137lb # [ 0.385387] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1138lb # [ 0.385851] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1139lb # [ 0.386034] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1140lb # [ 0.386050] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1141worker2 # [ 0.394996] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1142lb # [ 0.386079] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143worker2 # [ 0.395037] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144lb # [ 0.386531] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1145worker2 # [ 0.395524] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1146lb # [ 0.386715] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1147worker1 # [ 0.473666] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1148worker2 # [ 0.395715] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1149lb # [ 0.386731] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1150worker1 # [ 0.475875] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1151client # [ 0.394203] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1152worker2 # [ 0.395745] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153lb # [ 0.386760] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1154client # [ 0.394724] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1155lb # [ 0.386779] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1156worker2 # [ 0.396150] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1157client # [ 0.394927] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1158worker2 # [ 0.396331] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1159lb # [ 0.387243] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1160client # [ 0.394957] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1161lb # [ 0.387430] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1162worker2 # [ 0.396619] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1163worker2 # [ 0.396806] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1164client # [ 0.395401] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1165lb # [ 0.387460] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1166client # [ 0.395585] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1167worker2 # [ 0.396835] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1168lb # [ 0.387911] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1169client # [ 0.395833] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1170worker2 # [ 0.397283] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1171lb # [ 0.388098] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1172client # [ 0.396050] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1173worker2 # [ 0.397469] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1174lb # [ 0.388127] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1175client # [ 0.396089] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176worker2 # [ 0.397499] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1177lb # [ 0.388519] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1178client # [ 0.396565] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1179worker2 # [ 0.397949] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180lb # [ 0.388700] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1181client # [ 0.396752] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker2 # [ 0.398136] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1183lb # [ 0.389007] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1184client # [ 0.396782] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1185worker2 # [ 0.398166] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1186lb # [ 0.389195] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1187client # [ 0.397233] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1188worker2 # [ 0.398622] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1189lb # [ 0.389224] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190client # [ 0.397419] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1191worker2 # [ 0.398906] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1192worker1 # [ 0.486870] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1193lb # [ 0.389672] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1194worker2 # [ 0.398922] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1195client # [ 0.397449] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1196lb # [ 0.389857] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1197worker2 # [ 0.398951] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1198client # [ 0.397926] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1199lb # [ 0.389894] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200worker2 # [ 0.399404] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1201lb # [ 0.390353] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1202worker2 # [ 0.399586] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1203lb # [ 0.390540] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1204worker2 # [ 0.399601] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1205worker2 # [ 0.399631] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1206lb # [ 0.390569] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1207worker2 # [ 0.400209] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1208lb # [ 0.391023] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1209lb # [ 0.391307] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1210worker2 # [ 0.400220] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1211lb # [ 0.391324] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1212worker2 # [ 0.400225] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1213lb # [ 0.391354] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1214worker2 # [ 0.400268] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1215lb # [ 0.391811] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1216worker1 # [ 0.492741] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1217worker2 # [ 0.400314] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1218lb # [ 0.391993] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1219worker1 # [ 0.494695] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1220lb # [ 0.392009] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1221worker2 # [ 0.400361] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1222lb # [ 0.392038] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1223worker2 # [ 0.400408] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1224lb # [ 0.392618] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1225worker2 # [ 0.400454] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1226lb # [ 0.392630] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1227worker2 # [ 0.400500] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1228lb # [ 0.392635] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1229worker2 # [ 0.400546] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1230client # [ 0.418295] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1231lb # [ 0.392680] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1232worker2 # [ 0.400591] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1233client # [ 0.418325] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1234lb # [ 0.392725] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1235client # [ 0.418355] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1236worker2 # [ 0.400637] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1237lb # [ 0.392771] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1238client # [ 0.418852] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1239worker2 # [ 0.400714] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1240client # [ 0.419047] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1241lb # [ 0.392817] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1242worker2 # [ 0.400765] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1243client # [ 0.419064] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1244worker2 # [ 0.400787] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1245client # [ 0.419094] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1246worker2 # [ 0.400808] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1247client # [ 0.419698] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1248worker2 # [ 0.400829] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1249worker1 # [ 0.504749] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1250client # [ 0.419709] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1251worker2 # [ 0.400850] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1252worker1 # [ 0.506747] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1253client # [ 0.419714] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1254worker2 # [ 0.400871] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1255worker1 # [ 0.508514] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1256worker2 # [ 0.400892] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1257client # [ 0.419761] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1258worker2 # [ 0.400916] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1259client # [ 0.419809] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1260worker2 # [ 0.400938] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1261client # [ 0.419856] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1262worker2 # [ 0.400959] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1263client # [ 0.419905] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1264worker2 # [ 0.400981] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1265client # [ 0.419953] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1266worker2 # [ 0.401003] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1267worker2 # [ 0.401024] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1268client # [ 0.420011] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1269worker2 # [ 0.401045] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1270client # [ 0.420060] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1271worker2 # [ 0.401066] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1272client # [ 0.420107] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1273worker2 # [ 0.401086] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1274worker2 # [ 0.401107] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1275client # [ 0.420158] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1276worker2 # [ 0.401128] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1277client # [ 0.420288] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1278worker2 # [ 0.401153] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1279worker1 # [ 0.518642] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1280client # [ 0.420357] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1281worker2 # [ 0.401162] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1282lb # [ 0.440955] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1283client # [ 0.420379] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1284worker2 # [ 0.401167] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1285client # [ 0.420403] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1286lb # [ 0.441023] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1287worker2 # [ 0.401965] pci 0000:00:07.0: enabling device (0000 -> 0002)1288client # [ 0.420424] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1289lb # [ 0.441074] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1290client # [ 0.420446] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1291lb # [ 0.441125] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1292client # [ 0.420470] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1293lb # [ 0.441173] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1294client # [ 0.420492] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1295lb # [ 0.441318] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1296client # [ 0.420520] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1297lb # [ 0.441369] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1298client # [ 0.420542] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1299lb # [ 0.441393] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1300client # [ 0.420565] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1301lb # [ 0.441416] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1302client # [ 0.420587] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1303worker1 # [ 0.524669] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1304lb # [ 0.441438] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1305client # [ 0.420610] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1306worker1 # [ 0.527034] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1307client # [ 0.420632] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1308lb # [ 0.441461] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1309client # [ 0.420653] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1310lb # [ 0.441484] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1311client # [ 0.420674] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1312lb # [ 0.441506] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1313client # [ 0.420695] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1314lb # [ 0.441534] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1315client # [ 0.420716] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1316lb # [ 0.441555] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1317client # [ 0.420737] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1318lb # [ 0.441577] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1319client # [ 0.420765] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1320lb # [ 0.441599] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1321client # [ 0.420774] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1322lb # [ 0.441622] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1323client # [ 0.420778] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1324lb # [ 0.441643] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1325client # [ 0.421600] pci 0000:00:07.0: enabling device (0000 -> 0002)1326lb # [ 0.441664] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1327lb # [ 0.441685] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1328lb # [ 0.441706] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1329lb # [ 0.441727] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1330lb # [ 0.441748] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1331lb # [ 0.441782] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1332lb # [ 0.441792] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1333lb # [ 0.441797] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1334lb # [ 0.442637] pci 0000:00:07.0: enabling device (0000 -> 0002)1335worker1 # [ 0.545865] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1336worker2 # [ 0.478466] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1337worker1 # [ 0.548686] msm_serial: driver initialized1338worker1 # [ 0.548823] SuperH (H)SCI(F) driver initialized1339worker1 # [ 0.548878] STM32 USART driver initialized1340client # [ 0.477224] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1341worker2 # [ 0.488298] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1342lb # [ 0.487007] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1343worker2 # [ 0.492847] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1344worker2 # [ 0.494793] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1345client # [ 0.487631] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1346lb # [ 0.497111] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1347lb # [ 0.500779] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1348worker2 # [ 0.508536] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1349worker1 # [ 0.581806] loop: module loaded1350worker1 # [ 0.582016] virtio_blk virtio2: 1/0/0 default/read/poll queues1351worker2 # [ 0.510676] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1352worker1 # [ 0.582814] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1353client # [ 0.498844] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1354client # [ 0.500925] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1355worker2 # [ 0.514098] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1356client # [ 0.503021] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1357client # [ 0.505237] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1358lb # [ 0.510314] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1359worker2 # [ 0.516832] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1360worker1 # [ 0.589271] megasas: 07.734.00.00-rc11361lb # [ 0.514961] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1362worker1 # [ 0.590004] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1363worker1 # [ 0.592395] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001364worker1 # [ 0.592465] Intel/Sharp Extended Query Table at 0x00311365client # [ 0.515463] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1366client # [ 0.517325] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1367worker2 # [ 0.527048] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1368lb # [ 0.521026] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1369worker2 # [ 0.529002] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1370lb # [ 0.522848] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1371worker1 # [ 0.601632] Using buffer write method1372lb # [ 0.524734] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1373worker1 # [ 0.601720] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1374client # [ 0.528463] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1375client # [ 0.531243] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1376client # [ 0.533579] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1377worker1 # [ 0.612621] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001378worker2 # [ 0.539090] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1379worker1 # [ 0.612657] Intel/Sharp Extended Query Table at 0x00311380worker1 # [ 0.614315] Using buffer write method1381worker1 # [ 0.614350] Concatenating MTD devices:1382worker1 # [ 0.614355] (0): "0.flash"1383worker1 # [ 0.614365] (1): "0.flash"1384worker1 # [ 0.614369] into device "0.flash"1385lb # [ 0.538417] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1386lb # [ 0.540346] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1387worker2 # [ 0.552152] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1388client # [ 0.546457] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1389lb # [ 0.552179] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1390worker2 # [ 0.554951] msm_serial: driver initialized1391worker2 # [ 0.555099] SuperH (H)SCI(F) driver initialized1392worker2 # [ 0.555152] STM32 USART driver initialized1393client # [ 0.555604] msm_serial: driver initialized1394client # [ 0.555768] SuperH (H)SCI(F) driver initialized1395client # [ 0.555822] STM32 USART driver initialized1396lb # [ 0.562102] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1397lb # [ 0.564953] msm_serial: driver initialized1398lb # [ 0.565108] SuperH (H)SCI(F) driver initialized1399lb # [ 0.565165] STM32 USART driver initialized1400worker2 # [ 0.587446] loop: module loaded1401worker2 # [ 0.587641] virtio_blk virtio2: 1/0/0 default/read/poll queues1402worker2 # [ 0.588422] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1403client # [ 0.585011] loop: module loaded1404client # [ 0.585201] virtio_blk virtio2: 1/0/0 default/read/poll queues1405client # [ 0.586036] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1406worker2 # [ 0.595511] megasas: 07.734.00.00-rc11407worker2 # [ 0.596459] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1408worker2 # [ 0.598690] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001409worker2 # [ 0.598716] Intel/Sharp Extended Query Table at 0x00311410lb # [ 0.599607] loop: module loaded1411worker2 # [ 0.600411] Using buffer write method1412lb # [ 0.599795] virtio_blk virtio2: 1/0/0 default/read/poll queues1413worker2 # [ 0.600483] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1414lb # [ 0.600605] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1415worker2 # [ 0.602216] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001416worker2 # [ 0.602240] Intel/Sharp Extended Query Table at 0x00311417client # [ 0.598879] megasas: 07.734.00.00-rc11418client # [ 0.599589] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1419client # [ 0.601860] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001420client # [ 0.601887] Intel/Sharp Extended Query Table at 0x00311421client # [ 0.603435] Using buffer write method1422client # [ 0.603498] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1423client # [ 0.605283] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001424worker2 # [ 0.620872] Using buffer write method1425client # [ 0.605307] Intel/Sharp Extended Query Table at 0x00311426worker2 # [ 0.620908] Concatenating MTD devices:1427worker2 # [ 0.620912] (0): "0.flash"1428worker2 # [ 0.620916] (1): "0.flash"1429worker2 # [ 0.620924] into device "0.flash"1430lb # [ 0.613516] megasas: 07.734.00.00-rc11431lb # [ 0.614234] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1432client # [ 0.618771] Using buffer write method1433client # [ 0.618797] Concatenating MTD devices:1434client # [ 0.618801] (0): "0.flash"1435lb # [ 0.616430] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001436client # [ 0.618805] (1): "0.flash"1437client # [ 0.618812] into device "0.flash"1438lb # [ 0.616454] Intel/Sharp Extended Query Table at 0x00311439lb # [ 0.618022] Using buffer write method1440lb # [ 0.618087] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1441lb # [ 0.619722] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001442lb # [ 0.619745] Intel/Sharp Extended Query Table at 0x00311443lb # [ 0.638469] Using buffer write method1444lb # [ 0.638505] Concatenating MTD devices:1445lb # [ 0.638510] (0): "0.flash"1446lb # [ 0.638514] (1): "0.flash"1447lb # [ 0.638524] into device "0.flash"1448worker1 # [ 0.882601] Freeing initrd memory: 26976K1449worker1 # [ 0.888899] tun: Universal TUN/TAP device driver, 1.61450worker1 # [ 0.892737] thunder_xcv, ver 1.01451worker1 # [ 0.892782] thunder_bgx, ver 1.01452worker1 # [ 0.892805] nicpf, ver 1.01453worker1 # [ 0.893369] e1000: Intel(R) PRO/1000 Network Driver1454worker1 # [ 0.893376] e1000: Copyright (c) 1999-2006 Intel Corporation.1455worker1 # [ 0.893402] e1000e: Intel(R) PRO/1000 Network Driver1456worker1 # [ 0.893411] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457worker1 # [ 0.893440] igb: Intel(R) Gigabit Ethernet Network Driver1458worker1 # [ 0.893446] igb: Copyright (c) 2007-2014 Intel Corporation.1459worker1 # [ 0.893472] igbvf: Intel(R) Gigabit Virtual Function Network Driver1460worker1 # [ 0.893478] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1461worker1 # [ 0.893614] sky2: driver version 1.301462worker1 # [ 0.895230] usbcore: registered new interface driver usb-storage1463worker1 # [ 0.895314] usbcore: registered new interface driver usbserial_generic1464worker1 # [ 0.895330] usbserial: USB Serial support registered for generic1465worker1 # [ 0.896009] hv_vmbus: registering driver hyperv_keyboard1466worker1 # [ 0.897338] ehci-pci 0000:00:07.0: EHCI Host Controller1467worker1 # [ 0.897369] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11468worker1 # [ 0.897597] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001469worker1 # [ 0.909739] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001470worker1 # [ 0.910079] hub 1-0:1.0: USB hub found1471worker1 # [ 0.910100] hub 1-0:1.0: 6 ports detected1472worker1 # [ 0.912416] rtc-pl031 9010000.pl031: registered as rtc01473worker1 # [ 0.912444] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:00 UTC (1790141520)1474worker1 # [ 0.914491] i2c_dev: i2c /dev entries driver1475worker1 # [ 0.918146] sdhci: Secure Digital Host Controller Interface driver1476worker1 # [ 0.918158] sdhci: Copyright(c) Pierre Ossman1477worker1 # [ 0.918418] Synopsys Designware Multimedia Card Interface Driver1478worker1 # [ 0.918793] sdhci-pltfm: SDHCI platform and OF driver helper1479worker1 # [ 0.920329] hid: raw HID events driver (C) Jiri Kosina1480worker1 # [ 0.923850] usbcore: registered new interface driver usbhid1481worker1 # [ 0.923860] usbhid: USB HID core driver1482worker1 # [ 0.926131] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1483worker1 # [ 0.927654] drop_monitor: Initializing network drop monitor service1484worker1 # [ 0.927831] NET: Registered PF_INET6 protocol family1485worker1 # [ 0.930851] Segment Routing with IPv61486worker1 # [ 0.930867] In-situ OAM (IOAM) with IPv61487worker1 # [ 0.930895] NET: Registered PF_PACKET protocol family1488worker1 # [ 0.932504] 9pnet: Installing 9P2000 support1489worker1 # [ 0.932550] Key type dns_resolver registered1490worker1 # [ 0.939687] registered taskstats version 11491worker1 # [ 0.939873] Loading compiled-in X.509 certificates1492worker2 # [ 0.871288] Freeing initrd memory: 26944K1493worker1 # [ 0.948495] Demotion targets for Node 0: null1494worker2 # [ 0.877324] tun: Universal TUN/TAP device driver, 1.61495worker1 # [ 0.949333] Key type .fscrypt registered1496worker1 # [ 0.949344] Key type fscrypt-provisioning registered1497worker1 # [ 0.949445] ima: No TPM chip found, activating TPM-bypass!1498worker1 # [ 0.949465] ima: Allocated hash algorithm: sha11499worker1 # [ 0.949487] ima: No architecture policies found1500worker1 # [ 0.952962] input: gpio-keys as /devices/platform/gpio-keys/input/input01501worker2 # [ 0.881008] thunder_xcv, ver 1.01502worker2 # [ 0.881047] thunder_bgx, ver 1.01503worker2 # [ 0.881070] nicpf, ver 1.01504worker2 # [ 0.881621] e1000: Intel(R) PRO/1000 Network Driver1505worker2 # [ 0.881627] e1000: Copyright (c) 1999-2006 Intel Corporation.1506client # [ 0.877325] Freeing initrd memory: 26972K1507worker2 # [ 0.881651] e1000e: Intel(R) PRO/1000 Network Driver1508worker2 # [ 0.881660] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1509worker2 # [ 0.881706] igb: Intel(R) Gigabit Ethernet Network Driver1510worker2 # [ 0.881712] igb: Copyright (c) 2007-2014 Intel Corporation.1511worker2 # [ 0.881748] igbvf: Intel(R) Gigabit Virtual Function Network Driver1512worker2 # [ 0.881754] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1513worker2 # [ 0.881890] sky2: driver version 1.301514lb # [ 0.886284] Freeing initrd memory: 26948K1515worker2 # [ 0.890390] usbcore: registered new interface driver usb-storage1516client # [ 0.883433] tun: Universal TUN/TAP device driver, 1.61517worker2 # [ 0.890482] usbcore: registered new interface driver usbserial_generic1518worker2 # [ 0.890496] usbserial: USB Serial support registered for generic1519worker2 # [ 0.893760] ehci-pci 0000:00:07.0: EHCI Host Controller1520worker2 # [ 0.893788] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11521worker2 # [ 0.894053] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001522lb # [ 0.892349] tun: Universal TUN/TAP device driver, 1.61523worker2 # [ 0.896764] hv_vmbus: registering driver hyperv_keyboard1524worker2 # [ 0.898271] rtc-pl031 9010000.pl031: registered as rtc01525client # [ 0.887259] thunder_xcv, ver 1.01526client # [ 0.887301] thunder_bgx, ver 1.01527client # [ 0.887322] nicpf, ver 1.01528worker2 # [ 0.898296] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:00 UTC (1790141520)1529client # [ 0.887873] e1000: Intel(R) PRO/1000 Network Driver1530worker2 # [ 0.898597] i2c_dev: i2c /dev entries driver1531worker1 # [ 0.971552] clk: Disabling unused clocks1532client # [ 0.887880] e1000: Copyright (c) 1999-2006 Intel Corporation.1533worker1 # [ 0.971595] PM: genpd: Disabling unused power domains1534lb # [ 0.896066] thunder_xcv, ver 1.01535client # [ 0.887904] e1000e: Intel(R) PRO/1000 Network Driver1536lb # [ 0.896110] thunder_bgx, ver 1.01537client # [ 0.887912] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1538lb # [ 0.896133] nicpf, ver 1.01539lb # [ 0.896689] e1000: Intel(R) PRO/1000 Network Driver1540client # [ 0.887940] igb: Intel(R) Gigabit Ethernet Network Driver1541lb # [ 0.896696] e1000: Copyright (c) 1999-2006 Intel Corporation.1542client # [ 0.887945] igb: Copyright (c) 2007-2014 Intel Corporation.1543lb # [ 0.896721] e1000e: Intel(R) PRO/1000 Network Driver1544client # [ 0.888004] igbvf: Intel(R) Gigabit Virtual Function Network Driver1545worker1 # [ 0.975751] Freeing unused kernel memory: 4736K1546lb # [ 0.896729] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1547worker1 # [ 0.975969] Run /init as init process1548client # [ 0.888010] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1549lb # [ 0.896760] igb: Intel(R) Gigabit Ethernet Network Driver1550client # [ 0.888148] sky2: driver version 1.301551lb # [ 0.896765] igb: Copyright (c) 2007-2014 Intel Corporation.1552client # [ 0.889738] usbcore: registered new interface driver usb-storage1553lb # [ 0.896804] igbvf: Intel(R) Gigabit Virtual Function Network Driver1554client # [ 0.889785] usbcore: registered new interface driver usbserial_generic1555worker2 # [ 0.904033] sdhci: Secure Digital Host Controller Interface driver1556worker2 # [ 0.904044] sdhci: Copyright(c) Pierre Ossman1557lb # [ 0.896810] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1558client # [ 0.889798] usbserial: USB Serial support registered for generic1559worker2 # [ 0.904306] Synopsys Designware Multimedia Card Interface Driver1560client # [ 0.890883] ehci-pci 0000:00:07.0: EHCI Host Controller1561worker2 # [ 0.904674] sdhci-pltfm: SDHCI platform and OF driver helper1562client # [ 0.890907] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11563lb # [ 0.903601] sky2: driver version 1.301564worker2 # [ 0.906163] hid: raw HID events driver (C) Jiri Kosina1565client # [ 0.891186] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001566worker2 # [ 0.906410] usbcore: registered new interface driver usbhid1567worker2 # [ 0.906416] usbhid: USB HID core driver1568lb # [ 0.905940] ehci-pci 0000:00:07.0: EHCI Host Controller1569client # [ 0.902688] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001570client # [ 0.902993] hub 1-0:1.0: USB hub found1571lb # [ 0.905968] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11572worker2 # [ 0.911294] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001573client # [ 0.903019] hub 1-0:1.0: 6 ports detected1574worker2 # [ 0.911608] hub 1-0:1.0: USB hub found1575lb # [ 0.906193] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001576worker2 # [ 0.911627] hub 1-0:1.0: 6 ports detected1577client # [ 0.905247] hv_vmbus: registering driver hyperv_keyboard1578lb # [ 0.908925] usbcore: registered new interface driver usb-storage1579worker2 # [ 0.914278] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1580client # [ 0.906820] rtc-pl031 9010000.pl031: registered as rtc01581lb # [ 0.909016] usbcore: registered new interface driver usbserial_generic1582lb # [ 0.909031] usbserial: USB Serial support registered for generic1583client # [ 0.906844] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:00 UTC (1790141520)1584lb # [ 0.909636] hv_vmbus: registering driver hyperv_keyboard1585client # [ 0.907144] i2c_dev: i2c /dev entries driver1586worker2 # [ 0.916919] drop_monitor: Initializing network drop monitor service1587worker2 # [ 0.917064] NET: Registered PF_INET6 protocol family1588worker2 # [ 0.918952] Segment Routing with IPv61589worker1 # [ 0.990795] systemd[1]: Successfully made /usr/ read-only.1590lb # [ 0.913398] rtc-pl031 9010000.pl031: registered as rtc01591worker2 # [ 0.918980] In-situ OAM (IOAM) with IPv61592worker2 # [ 0.919007] NET: Registered PF_PACKET protocol family1593lb # [ 0.913422] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:32:00 UTC (1790141520)1594lb # [ 0.913736] i2c_dev: i2c /dev entries driver1595client # [ 0.912378] sdhci: Secure Digital Host Controller Interface driver1596client # [ 0.912388] sdhci: Copyright(c) Pierre Ossman1597worker2 # [ 0.920626] 9pnet: Installing 9P2000 support1598worker2 # [ 0.920675] Key type dns_resolver registered1599client # [ 0.912652] Synopsys Designware Multimedia Card Interface Driver1600client # [ 0.913031] sdhci-pltfm: SDHCI platform and OF driver helper1601lb # [ 0.919148] sdhci: Secure Digital Host Controller Interface driver1602client # [ 0.917270] hid: raw HID events driver (C) Jiri Kosina1603lb # [ 0.919160] sdhci: Copyright(c) Pierre Ossman1604client # [ 0.917506] usbcore: registered new interface driver usbhid1605lb # [ 0.919431] Synopsys Designware Multimedia Card Interface Driver1606client # [ 0.917512] usbhid: USB HID core driver1607lb # [ 0.919807] sdhci-pltfm: SDHCI platform and OF driver helper1608lb # [ 0.923556] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001609lb # [ 0.924578] hub 1-0:1.0: USB hub found1610lb # [ 0.925092] hub 1-0:1.0: 6 ports detected1611client # [ 0.920418] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1612worker2 # [ 0.927551] registered taskstats version 11613client # [ 0.921939] drop_monitor: Initializing network drop monitor service1614worker2 # [ 0.927701] Loading compiled-in X.509 certificates1615client # [ 0.922079] NET: Registered PF_INET6 protocol family1616lb # [ 0.926490] hid: raw HID events driver (C) Jiri Kosina1617client # [ 0.925208] Segment Routing with IPv61618lb # [ 0.926751] usbcore: registered new interface driver usbhid1619client # [ 0.925226] In-situ OAM (IOAM) with IPv61620lb # [ 0.926757] usbhid: USB HID core driver1621client # [ 0.925253] NET: Registered PF_PACKET protocol family1622lb # [ 0.929725] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1623worker2 # [ 0.936453] Demotion targets for Node 0: null1624worker2 # [ 0.936557] Key type .fscrypt registered1625lb # [ 0.931310] drop_monitor: Initializing network drop monitor service1626client # [ 0.927100] 9pnet: Installing 9P2000 support1627worker2 # [ 0.936564] Key type fscrypt-provisioning registered1628lb # [ 0.931444] NET: Registered PF_INET6 protocol family1629client # [ 0.927146] Key type dns_resolver registered1630worker2 # [ 0.936654] ima: No TPM chip found, activating TPM-bypass!1631lb # [ 0.934458] Segment Routing with IPv61632worker2 # [ 0.936673] ima: Allocated hash algorithm: sha11633lb # [ 0.934488] In-situ OAM (IOAM) with IPv61634worker2 # [ 0.936693] ima: No architecture policies found1635lb # [ 0.934521] NET: Registered PF_PACKET protocol family1636worker2 # [ 0.940694] input: gpio-keys as /devices/platform/gpio-keys/input/input01637lb # [ 0.936174] 9pnet: Installing 9P2000 support1638client # [ 0.933603] registered taskstats version 11639lb # [ 0.936217] Key type dns_resolver registered1640client # [ 0.933746] Loading compiled-in X.509 certificates1641lb # [ 0.942974] registered taskstats version 11642lb # [ 0.943134] Loading compiled-in X.509 certificates1643client # [ 0.942340] Demotion targets for Node 0: null1644client # [ 0.942446] Key type .fscrypt registered1645client # [ 0.942452] Key type fscrypt-provisioning registered1646client # [ 0.942547] ima: No TPM chip found, activating TPM-bypass!1647client # [ 0.942566] ima: Allocated hash algorithm: sha11648client # [ 0.942591] ima: No architecture policies found1649client # [ 0.946717] input: gpio-keys as /devices/platform/gpio-keys/input/input01650lb # [ 0.951876] Demotion targets for Node 0: null1651lb # [ 0.951992] Key type .fscrypt registered1652lb # [ 0.951997] Key type fscrypt-provisioning registered1653lb # [ 0.952094] ima: No TPM chip found, activating TPM-bypass!1654worker2 # [ 0.958553] clk: Disabling unused clocks1655lb # [ 0.952114] ima: Allocated hash algorithm: sha11656worker2 # [ 0.958579] PM: genpd: Disabling unused power domains1657lb # [ 0.952135] ima: No architecture policies found1658lb # [ 0.956184] input: gpio-keys as /devices/platform/gpio-keys/input/input01659worker2 # [ 0.962731] Freeing unused kernel memory: 4736K1660worker2 # [ 0.963529] Run /init as init process1661client # [ 0.964967] clk: Disabling unused clocks1662client # [ 0.964993] PM: genpd: Disabling unused power domains1663worker2 # [ 0.976998] systemd[1]: Successfully made /usr/ read-only.1664client # [ 0.969117] Freeing unused kernel memory: 4736K1665client # [ 0.969332] Run /init as init process1666lb # [ 0.974403] clk: Disabling unused clocks1667lb # [ 0.974429] PM: genpd: Disabling unused power domains1668lb # [ 0.978590] Freeing unused kernel memory: 4736K1669lb # [ 0.978787] Run /init as init process1670client # [ 0.984333] systemd[1]: Successfully made /usr/ read-only.1671lb # [ 0.994997] systemd[1]: Successfully made /usr/ read-only.1672worker1 # [ 1.156643] usb 1-1: new high-speed USB device number 2 using ehci-pci1673client # [ 1.150253] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker2 # [ 1.158894] usb 1-1: new high-speed USB device number 2 using ehci-pci1675lb # [ 1.172913] usb 1-1: new high-speed USB device number 2 using ehci-pci1676worker1 # [ 1.309195] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677worker1 # [ 1.325604] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678worker1 # [ 1.337611] systemd[1]: Detected virtualization qemu.1679worker1 # [ 1.339584] systemd[1]: Detected architecture arm64.1680worker1 # [ 1.341520] systemd[1]: Running in initrd.1681worker1 # [ 1.344071] systemd[1]: Initializing machine ID from random generator.1682worker1 # [ 1.347121] systemd[1]: Hostname set to <worker1>.1683client # [ 1.302699] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684worker2 # [ 1.311490] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685worker1 # [ 1.396850] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01686worker2 # [ 1.317667] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1687worker2 # [ 1.330121] systemd[1]: Detected virtualization qemu.1688worker2 # [ 1.332305] systemd[1]: Detected architecture arm64.1689lb # [ 1.325455] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11690worker2 # [ 1.334211] systemd[1]: Running in initrd.1691client # [ 1.318973] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692worker2 # [ 1.336851] systemd[1]: Initializing machine ID from random generator.1693client # [ 1.331093] systemd[1]: Detected virtualization qemu.1694worker2 # [ 1.339726] systemd[1]: Hostname set to <worker2>.1695client # [ 1.333096] systemd[1]: Detected architecture arm64.1696client # [ 1.335050] systemd[1]: Running in initrd.1697client # [ 1.337617] systemd[1]: Initializing machine ID from random generator.1698lb # [ 1.331572] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1699client # [ 1.340523] systemd[1]: Hostname set to <client>.1700lb # [ 1.343997] systemd[1]: Detected virtualization qemu.1701lb # [ 1.344090] systemd[1]: Detected architecture arm64.1702lb # [ 1.344114] systemd[1]: Running in initrd.1703lb # [ 1.350580] systemd[1]: Initializing machine ID from random generator.1704lb # [ 1.353346] systemd[1]: Hostname set to <lb>.1705client # [ 1.386453] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker2 # [ 1.395068] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707lb # [ 1.405092] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708worker1 # [ 1.516650] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.514867] usb 1-2: new high-speed USB device number 3 using ehci-pci1710client # [ 1.510214] usb 1-2: new high-speed USB device number 3 using ehci-pci1711lb # [ 1.524886] usb 1-2: new high-speed USB device number 3 using ehci-pci1712worker1 # [ 1.645852] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713worker1 # [ 1.677460] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21714worker1 # [ 1.683073] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01715worker2 # [ 1.640438] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716client # [ 1.643763] systemd[1]: bpf-restrict-fs: LSM BPF program attached1717lb # [ 1.655330] systemd[1]: bpf-restrict-fs: LSM BPF program attached1718worker2 # [ 1.671655] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719worker2 # [ 1.676769] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720client # [ 1.670995] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721client # [ 1.678493] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722worker1 # [ 1.764448] systemd[1]: Queued start job for default target Initrd Default Target.1723lb # [ 1.685640] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21724lb # [ 1.691282] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01725worker1 # [ 1.775022] systemd[1]: Created slice Slice /system/modprobe.1726worker1 # [ 1.776256] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1727worker1 # [ 1.777723] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728worker1 # [ 1.778821] systemd[1]: Reached target Path Units.1729worker1 # [ 1.779669] systemd[1]: Reached target Slice Units.1730worker1 # [ 1.780523] systemd[1]: Reached target Swaps.1731worker1 # [ 1.781364] systemd[1]: Reached target Timer Units.1732worker1 # [ 1.782398] systemd[1]: Listening on D-Bus System Message Bus Socket.1733worker1 # [ 1.783681] systemd[1]: Listening on Journal Socket (/dev/log).1734worker1 # [ 1.784987] systemd[1]: Listening on Journal Sockets.1735worker1 # [ 1.785130] systemd[1]: Listening on udev Control Socket.1736worker1 # [ 1.785246] systemd[1]: Listening on udev Kernel Socket.1737worker1 # [ 1.785269] systemd[1]: Reached target Socket Units.1738worker1 # [ 1.790353] systemd[1]: Starting Create List of Static Device Nodes...1739worker1 # [ 1.791532] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1740worker1 # [ 1.800749] systemd[1]: Mounting Kernel Configuration File System...1741worker1 # [ 1.808892] systemd[1]: Starting Journal Service...1742worker2 # [ 1.754068] systemd[1]: Queued start job for default target Initrd Default Target.1743worker1 # [ 1.832724] systemd[1]: Starting Load Kernel Modules...1744worker1 # [ 1.833683] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1745worker2 # [ 1.765315] systemd[1]: Created slice Slice /system/modprobe.1746worker2 # [ 1.766526] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1747client # [ 1.759049] systemd[1]: Queued start job for default target Initrd Default Target.1748worker2 # [ 1.768014] systemd[1]: Expecting device /dev/disk/by-label/nixos...1749worker2 # [ 1.769108] systemd[1]: Reached target Path Units.1750worker2 # [ 1.769944] systemd[1]: Reached target Slice Units.1751worker2 # [ 1.770795] systemd[1]: Reached target Swaps.1752worker2 # [ 1.771639] systemd[1]: Reached target Timer Units.1753worker2 # [ 1.772668] systemd[1]: Listening on D-Bus System Message Bus Socket.1754worker2 # [ 1.773923] systemd[1]: Listening on Journal Socket (/dev/log).1755client # [ 1.768610] systemd[1]: Created slice Slice /system/modprobe.1756worker2 # [ 1.775113] systemd[1]: Listening on Journal Sockets.1757lb # [ 1.772401] systemd[1]: Queued start job for default target Initrd Default Target.1758worker2 # [ 1.775256] systemd[1]: Listening on udev Control Socket.1759client # [ 1.769831] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1760worker2 # [ 1.775375] systemd[1]: Listening on udev Kernel Socket.1761worker2 # [ 1.775398] systemd[1]: Reached target Socket Units.1762client # [ 1.771279] systemd[1]: Expecting device /dev/disk/by-label/nixos...1763client # [ 1.772394] systemd[1]: Reached target Path Units.1764worker2 # [ 1.780519] systemd[1]: Starting Create List of Static Device Nodes...1765worker1 # [ 1.852838] systemd[1]: Starting Coldplug All udev Devices...1766client # [ 1.773245] systemd[1]: Reached target Slice Units.1767client # [ 1.774100] systemd[1]: Reached target Swaps.1768worker2 # [ 1.781715] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1769client # [ 1.774949] systemd[1]: Reached target Timer Units.1770lb # [ 1.778983] systemd[1]: Created slice Slice /system/modprobe.1771client # [ 1.775999] systemd[1]: Listening on D-Bus System Message Bus Socket.1772lb # [ 1.780105] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1773client # [ 1.777283] systemd[1]: Listening on Journal Socket (/dev/log).1774worker1 # [ 1.857740] systemd-journald[72]: Collecting audit messages is disabled.1775client # [ 1.778521] systemd[1]: Listening on Journal Sockets.1776lb # [ 1.781455] systemd[1]: Expecting device /dev/disk/by-label/nixos...1777client # [ 1.778664] systemd[1]: Listening on udev Control Socket.1778lb # [ 1.781489] systemd[1]: Reached target Path Units.1779lb # [ 1.781507] systemd[1]: Reached target Slice Units.1780client # [ 1.778780] systemd[1]: Listening on udev Kernel Socket.1781lb # [ 1.781527] systemd[1]: Reached target Swaps.1782client # [ 1.778805] systemd[1]: Reached target Socket Units.1783lb # [ 1.781546] systemd[1]: Reached target Timer Units.1784lb # [ 1.781732] systemd[1]: Listening on D-Bus System Message Bus Socket.1785client # [ 1.783900] systemd[1]: Starting Create List of Static Device Nodes...1786lb # [ 1.781870] systemd[1]: Listening on Journal Socket (/dev/log).1787worker2 # [ 1.790979] systemd[1]: Mounting Kernel Configuration File System...1788lb # [ 1.781998] systemd[1]: Listening on Journal Sockets.1789client # [ 1.785105] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1790lb # [ 1.782116] systemd[1]: Listening on udev Control Socket.1791lb # [ 1.782229] systemd[1]: Listening on udev Kernel Socket.1792lb # [ 1.782249] systemd[1]: Reached target Socket Units.1793lb # [ 1.792316] systemd[1]: Starting Create List of Static Device Nodes...1794worker1 # [ 1.868718] systemd[1]: Finished Create List of Static Device Nodes.1795lb # [ 1.793429] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1796worker2 # [ 1.799123] systemd[1]: Starting Journal Service...1797worker1 # [ 1.869377] systemd[1]: Mounted Kernel Configuration File System.1798client # [ 1.794332] systemd[1]: Mounting Kernel Configuration File System...1799lb # [ 1.803924] systemd[1]: Mounting Kernel Configuration File System...1800client # [ 1.802525] systemd[1]: Starting Journal Service...1801worker1 # [ 1.884961] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1802lb # [ 1.811337] systemd[1]: Starting Journal Service...1803worker2 # [ 1.824270] systemd[1]: Starting Load Kernel Modules...1804worker2 # [ 1.824369] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1805worker2 # [ 1.828155] systemd[1]: Starting Coldplug All udev Devices...1806lb # [ 1.825468] systemd[1]: Starting Load Kernel Modules...1807lb # [ 1.825578] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1808client # [ 1.826785] systemd[1]: Starting Load Kernel Modules...1809client # [ 1.827783] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1810lb # [ 1.833175] systemd[1]: Starting Coldplug All udev Devices...1811worker2 # [ 1.847037] systemd[1]: Finished Create List of Static Device Nodes.1812worker1 # [ 1.921135] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1813worker1 # [ 1.923496] systemd[1]: Starting Create Static Device Nodes in /dev...1814lb # [ 1.848966] systemd[1]: Finished Create List of Static Device Nodes.1815lb # [ 1.849770] systemd[1]: Mounted Kernel Configuration File System.1816client # [ 1.850833] systemd[1]: Starting Coldplug All udev Devices...1817worker2 # [ 1.860015] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1818worker1 # [ 1.934063] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1819client # [ 1.855497] systemd-journald[72]: Collecting audit messages is disabled.1820worker2 # [ 1.872151] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1821worker2 # [ 1.874627] systemd[1]: Mounted Kernel Configuration File System.1822lb # [ 1.869247] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1823client # [ 1.870277] systemd[1]: Finished Create List of Static Device Nodes.1824worker1 # [ 1.944642] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1825client # [ 1.870993] systemd[1]: Mounted Kernel Configuration File System.1826worker2 # [ 1.880740] systemd-journald[72]: Collecting audit messages is disabled.1827worker1 # [ 1.950557] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01828worker1 # [ 1.950784] [drm] features: -virgl +edid -resource_blob -host_visible1829worker1 # [ 1.950795] [drm] features: -context_init1830worker1 # [ 1.951466] [drm] number of scanouts: 11831worker1 # [ 1.951484] [drm] number of cap sets: 01832client # [ 1.878317] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1833lb # [ 1.891910] systemd-journald[72]: Collecting audit messages is disabled.1834worker2 # [ 1.891389] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1835worker2 # [ 1.897422] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01836worker2 # [ 1.897644] [drm] features: -virgl +edid -resource_blob -host_visible1837worker2 # [ 1.897654] [drm] features: -context_init1838worker2 # [ 1.898326] [drm] number of scanouts: 11839worker2 # [ 1.898344] [drm] number of cap sets: 01840lb # [ 1.898898] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1841worker1 # [ 1.977161] systemd[1]: Finished Create Static Device Nodes in /dev.1842worker1 # [ 1.977461] systemd[1]: Reached target Preparation for Local File Systems.1843worker1 # [ 1.977487] systemd[1]: Reached target Local File Systems.1844client # [ 1.899641] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1845worker1 # [ 1.981328] systemd[1]: Starting Rule-based Manager for Device Events and Files...1846lb # [ 1.909376] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1847worker2 # [ 1.915447] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1848worker1 # [ 1.985842] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1849lb # [ 1.911691] systemd[1]: Starting Create Static Device Nodes in /dev...1850worker2 # [ 1.917791] systemd[1]: Starting Create Static Device Nodes in /dev...1851worker1 # [ 1.985860] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01852client # [ 1.914275] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1853lb # [ 1.918982] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1854client # [ 1.916944] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1855client # [ 1.919292] systemd[1]: Starting Create Static Device Nodes in /dev...1856lb # [ 1.929980] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01857lb # [ 1.930207] [drm] features: -virgl +edid -resource_blob -host_visible1858worker2 # [ 1.931038] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1859lb # [ 1.930216] [drm] features: -context_init1860worker2 # [ 1.931054] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01861worker1 # [ 2.005257] Console: switching to colour frame buffer device 160x501862lb # [ 1.930912] [drm] number of scanouts: 11863lb # [ 1.930930] [drm] number of cap sets: 01864client # [ 1.935255] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01865client # [ 1.935487] [drm] features: -virgl +edid -resource_blob -host_visible1866client # [ 1.935497] [drm] features: -context_init1867client # [ 1.936219] [drm] number of scanouts: 11868client # [ 1.936236] [drm] number of cap sets: 01869worker2 # [ 1.951187] Console: switching to colour frame buffer device 160x501870lb # [ 1.953367] systemd[1]: Finished Create Static Device Nodes in /dev.1871lb # [ 1.953533] systemd[1]: Reached target Preparation for Local File Systems.1872lb # [ 1.953558] systemd[1]: Reached target Local File Systems.1873worker1 # [ 2.033175] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1874lb # [ 1.957295] systemd[1]: Starting Rule-based Manager for Device Events and Files...1875client # [ 1.950508] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1876client # [ 1.950524] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01877worker1 # [ 2.038951] systemd[1]: Started Journal Service.1878lb # [ 1.962192] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1879lb # [ 1.962210] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01880worker2 # [ 1.971374] systemd[1]: Finished Create Static Device Nodes in /dev.1881worker2 # [ 1.971526] systemd[1]: Reached target Preparation for Local File Systems.1882worker2 # [ 1.971549] systemd[1]: Reached target Local File Systems.1883worker2 # [ 1.975273] systemd[1]: Starting Rule-based Manager for Device Events and Files...1884worker1 # [ 2.036440] systemd-modules-load[74]: Inserted module 'dm_mod'1885worker1 # [ 2.037664] systemd-modules-load[74]: Module 'virtio_balloon' is built in1886worker1 # [ 2.038709] systemd-modules-load[74]: Module 'virtio_console' is built in1887client # [ 1.974750] systemd[1]: Finished Create Static Device Nodes in /dev.1888worker1 # [ 2.039758] systemd-modules-load[74]: Inserted module 'virtio_gpu'1889client # [ 1.975064] systemd[1]: Reached target Preparation for Local File Systems.1890worker2 # [ 1.983434] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1891client # [ 1.975090] systemd[1]: Reached target Local File Systems.1892client # [ 1.981080] systemd[1]: Starting Rule-based Manager for Device Events and Files...1893lb # [ 1.981141] Console: switching to colour frame buffer device 160x501894worker2 # [ 1.995363] systemd[1]: Finished Load Kernel Modules.1895worker1 # [ 2.052591] systemd-modules-load[74]: Module 'virtio_rng' is built in1896worker1 # [ 2.053616] systemd[1]: Starting Create System Files and Directories...1897worker2 # [ 1.998305] systemd[1]: Starting Apply Kernel Variables...1898worker1 # [ 2.054638] systemd[1]: Finished Load Kernel Modules.1899client # [ 1.986565] Console: switching to colour frame buffer device 160x501900client # [ 1.993016] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1901worker1 # [ 2.067387] systemd[1]: Starting Apply Kernel Variables...1902lb # [ 2.016062] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1903client # [ 2.018469] systemd[1]: Finished Load Kernel Modules.1904client # [ 2.021427] systemd[1]: Starting Apply Kernel Variables...1905worker1 # [ 2.087248] systemd-udevd[79]: Using default interface naming scheme 'v261'.1906lb # [ 2.033419] systemd[1]: Finished Load Kernel Modules.1907worker1 # [ 2.096201] systemd[1]: Finished Create System Files and Directories.1908lb # [ 2.037191] systemd[1]: Starting Apply Kernel Variables...1909client # [ 2.034354] systemd[1]: Started Journal Service.1910client # [ 2.023073] systemd-modules-load[74]: Inserted module 'dm_mod'1911worker1 # [ 2.101274] systemd[1]: Finished Apply Kernel Variables.1912client # [ 2.024158] systemd-modules-load[74]: Module 'virtio_balloon' is built in1913worker2 # [ 2.051445] systemd[1]: Finished Apply Kernel Variables.1914client # [ 2.036317] systemd-modules-load[74]: Module 'virtio_console' is built in1915client # [ 2.037470] systemd-modules-load[74]: Inserted module 'virtio_gpu'1916worker2 # [ 2.057604] systemd[1]: Started Journal Service.1917client # [ 2.039597] systemd-modules-load[74]: Module 'virtio_rng' is built in1918client # [ 2.045102] systemd[1]: Starting Create System Files and Directories...1919worker1 # [ 2.123022] systemd[1]: Started Rule-based Manager for Device Events and Files.1920worker2 # [ 2.055829] systemd-modules-load[73]: Inserted module 'dm_mod'1921worker2 # [ 2.057128] systemd-modules-load[73]: Module 'virtio_balloon' is built in1922worker2 # [ 2.060908] systemd-modules-load[73]: Module 'virtio_console' is built in1923worker2 # [ 2.067690] systemd-modules-load[73]: Inserted module 'virtio_gpu'1924lb # [ 2.064900] systemd-modules-load[73]: Inserted module 'dm_mod'1925client # [ 2.065156] systemd[1]: Finished Apply Kernel Variables.1926worker2 # [ 2.073160] systemd-modules-load[73]: Module 'virtio_rng' is built in1927lb # [ 2.082800] systemd[1]: Started Journal Service.1928lb # [ 2.069046] systemd-modules-load[73]: Module 'virtio_balloon' is built in1929lb # [ 2.070150] systemd-modules-load[73]: Module 'virtio_console' is built in1930worker2 # [ 2.077534] systemd[1]: Starting Create System Files and Directories...1931lb # [ 2.071227] systemd-modules-load[73]: Inserted module 'virtio_gpu'1932client # [ 2.074523] systemd-udevd[78]: Using default interface naming scheme 'v261'.1933worker2 # [ 2.084645] systemd-udevd[79]: Using default interface naming scheme 'v261'.1934lb # [ 2.081080] systemd-modules-load[73]: Module 'virtio_rng' is built in1935lb # [ 2.082369] systemd-udevd[78]: Using default interface naming scheme 'v261'.1936lb # [ 2.083430] systemd[1]: Starting Create System Files and Directories...1937lb # [ 2.088379] systemd[1]: Finished Apply Kernel Variables.1938worker2 # [ 2.096124] systemd[1]: Finished Create System Files and Directories.1939client # [ 2.092995] systemd[1]: Finished Create System Files and Directories.1940worker2 # [ 2.107689] systemd[1]: Started Rule-based Manager for Device Events and Files.1941worker1 # [ 2.183558] systemd[1]: Starting Virtual Console Setup...1942lb # [ 2.108984] systemd[1]: Finished Create System Files and Directories.1943client # [ 2.116691] systemd[1]: Started Rule-based Manager for Device Events and Files.1944lb # [ 2.119813] systemd[1]: Started Rule-based Manager for Device Events and Files.1945worker1 # [ 2.224504] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1946worker1 # [ 2.227806] systemd[1]: Finished Virtual Console Setup.1947worker2 # [ 2.163709] systemd[1]: Starting Virtual Console Setup...1948client # [ 2.164864] systemd[1]: Starting Virtual Console Setup...1949lb # [ 2.178043] systemd[1]: Starting Virtual Console Setup...1950worker2 # [ 2.208643] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1951worker2 # [ 2.211621] systemd[1]: Finished Virtual Console Setup.1952client # [ 2.216408] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1953client # [ 2.219688] systemd[1]: Finished Virtual Console Setup.1954lb # [ 2.224457] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1955lb # [ 2.227619] systemd[1]: Finished Virtual Console Setup.1956worker1 # [ 2.835738] systemd[1]: Finished Coldplug All udev Devices.1957worker1 # [ 2.840185] systemd[1]: Reached target System Initialization.1958worker1 # [ 2.840988] systemd[1]: Reached target Basic System.1959worker2 # [ 2.817125] systemd[1]: Finished Coldplug All udev Devices.1960worker2 # [ 2.820117] systemd[1]: Reached target System Initialization.1961worker2 # [ 2.820998] systemd[1]: Reached target Basic System.1962lb # [ 2.818641] systemd[1]: Finished Coldplug All udev Devices.1963lb # [ 2.824182] systemd[1]: Reached target System Initialization.1964lb # [ 2.825200] systemd[1]: Reached target Basic System.1965client # [ 2.824854] systemd[1]: Finished Coldplug All udev Devices.1966client # [ 2.825802] systemd[1]: Reached target System Initialization.1967client # [ 2.826630] systemd[1]: Reached target Basic System.1968worker1 # [ 2.976291] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969worker1 # [ 3.003167] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1970worker2 # [ 2.948637] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 3.021166] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 2.975963] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973worker2 # [ 2.984256] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1974worker2 # [ 2.988384] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1975client # [ 2.983516] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976worker1 # [ 3.082333] systemd[1]: Found device /dev/disk/by-label/nixos.1977worker1 # [ 3.085278] systemd[1]: Reached target Initrd Root Device.1978worker1 # [ 3.088704] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1979lb # [ 3.012099] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1980client # [ 3.014537] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1981client # [ 3.017409] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1982lb # [ 3.024145] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1983worker2 # [ 3.049997] systemd[1]: Found device /dev/disk/by-label/nixos.1984worker2 # [ 3.051833] systemd[1]: Reached target Initrd Root Device.1985worker2 # [ 3.056077] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986worker1 # [ 3.136192] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1987worker1 # [ 3.144109] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1988worker1 # [ 3.148209] systemd[1]: Mounting /sysroot...1989client # [ 3.081715] systemd[1]: Found device /dev/disk/by-label/nixos.1990client # [ 3.086691] systemd[1]: Reached target Initrd Root Device.1991client # [ 3.092557] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992lb # [ 3.101046] systemd[1]: Found device /dev/disk/by-label/nixos.1993worker2 # [ 3.110828] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1994lb # [ 3.104858] systemd[1]: Reached target Initrd Root Device.1995lb # [ 3.107668] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1996worker1 # [ 3.202975] EXT4-fs (vda): mounted filesystem 07f97f2e-1a27-4afe-86d2-1a582037d5c9 r/w with ordered data mode. Quota mode: none.1997worker2 # [ 3.118483] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998worker1 # [ 3.191572] systemd[1]: Mounted /sysroot.1999worker1 # [ 3.192535] systemd[1]: Reached target Initrd Root File System.2000worker2 # [ 3.124127] systemd[1]: Mounting /sysroot...
loading 2,091 hidden lines…
4092worker2 # [ 56.143352] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.757Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4093worker2 # [ 56.146136] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.757Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4094lb # [ 56.171191] niks3-server[975]: 2026/09/23 05:32:55 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4095lb # [ 56.187189] niks3-server[975]: 2026/09/23 05:32:55 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4096lb # [ 56.204916] niks3-server[975]: 2026/09/23 05:32:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4097worker2 # [ 56.217299] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.832Z level=INFO msg="Uploading 1 narinfos"4098lb # [ 56.211315] niks3-server[975]: 2026/09/23 05:32:55 INFO Signed narinfos id=10 count=14099lb # [ 56.217990] niks3-server[975]: 2026/09/23 05:32:55 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4100lb # [ 56.232145] niks3-server[975]: 2026/09/23 05:32:55 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4101lb # [ 56.238835] niks3-server[975]: 2026/09/23 05:32:55 INFO Completed upload id=104102worker2 # [ 56.245338] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.861Z level=INFO msg="Upload complete. (275ms)"4103lb # [ 56.245222] niks3-server[975]: 2026/09/23 05:32:55 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4104worker2 # [ 56.261608] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14105worker1 # [ 56.339717] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04106worker1 # [ 56.346539] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14107worker2 # [ 56.283188] nix-daemon[839]: reaped child process 884, status = succeeded4108worker2 # [ 56.317249] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584109worker2 # [ 56.341164] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584110worker2 # [ 56.358634] nix-grpc-daemon[866]: time=2026-09-23T05:32:55.974Z level=INFO msg="All 1 paths already cached"4111client # copying 1 paths to 'lb:50051'...4112client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4113worker2 # [ 56.371479] nix-grpc-daemon[769]: ts=2026-09-23T05:32:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343584114worker2 # [ 56.386545] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4115worker2 # [ 56.552465] nix-daemon[839]: accepted connection from pid 909, user nix-grpc-daemon (trusted)4116worker2 # [ 56.575597] nix-daemon[839]: reaped child process 912, status = succeeded4117lb # [ 56.573609] niks3-server[975]: 2026/09/23 05:32:56 INFO Received uploads request method=POST path=/api/pending_closures4118worker2 # [ 56.592518] nix-grpc-daemon[866]: time=2026-09-23T05:32:56.208Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4119worker2 # [ 56.594375] nix-grpc-daemon[866]: time=2026-09-23T05:32:56.208Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4120lb # [ 56.613323] niks3-server[975]: 2026/09/23 05:32:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4121worker2 # [ 56.634382] nix-grpc-daemon[866]: time=2026-09-23T05:32:56.250Z level=INFO msg="Uploading 1 narinfos"4122lb # [ 56.634785] niks3-server[975]: 2026/09/23 05:32:56 INFO Signed narinfos id=11 count=14123lb # [ 56.643380] niks3-server[975]: 2026/09/23 05:32:56 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4124lb # [ 56.654243] niks3-server[975]: 2026/09/23 05:32:56 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4125lb # [ 56.667442] niks3-server[975]: 2026/09/23 05:32:56 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4126lb # [ 56.676380] niks3-server[975]: 2026/09/23 05:32:56 INFO Completed upload id=114127worker2 # [ 56.682340] nix-grpc-daemon[866]: time=2026-09-23T05:32:56.298Z level=INFO msg="Upload complete. (276ms)"4128lb # [ 56.679381] niks3-server[975]: 2026/09/23 05:32:56 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4129worker2 # [ 56.694611] nix-grpc-daemon[769]: ts=2026-09-23T05:32:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=6244130worker2 # [ 56.708739] nix-grpc-daemon[769]: ts=2026-09-23T05:32:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584131worker2 # [ 56.722852] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4132worker2 # [ 56.731407] nix-daemon[839]: reaped child process 907, status = succeeded4133client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4134worker2 # [ 56.747182] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4135client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4136client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4137client # farm-b> LOG-farm-b-sig4138worker2 # [ 58.054253] nix-daemon[839]: reaped child process 916, status = succeeded4139worker2 # [ 58.140994] nix-daemon[839]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4140worker2 # [ 58.163293] nix-daemon[839]: reaped child process 935, status = succeeded4141lb # [ 58.163245] niks3-server[975]: 2026/09/23 05:32:57 INFO Received uploads request method=POST path=/api/pending_closures4142worker2 # [ 58.180534] nix-grpc-daemon[866]: time=2026-09-23T05:32:57.796Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4143worker2 # [ 58.182173] nix-grpc-daemon[866]: time=2026-09-23T05:32:57.796Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4144lb # [ 58.222255] niks3-server[975]: 2026/09/23 05:32:57 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4145worker2 # [ 58.239927] nix-grpc-daemon[866]: time=2026-09-23T05:32:57.855Z level=INFO msg="Uploading 1 narinfos"4146lb # [ 58.234336] niks3-server[975]: 2026/09/23 05:32:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4147lb # [ 58.244333] niks3-server[975]: 2026/09/23 05:32:57 INFO Signed narinfos id=12 count=14148lb # [ 58.246945] niks3-server[975]: 2026/09/23 05:32:57 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4149lb # [ 58.260177] niks3-server[975]: 2026/09/23 05:32:57 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4150lb # [ 58.273506] niks3-server[975]: 2026/09/23 05:32:57 INFO Completed upload id=124151lb # [ 58.275296] niks3-server[975]: 2026/09/23 05:32:57 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4152worker2 # [ 58.280561] nix-grpc-daemon[866]: time=2026-09-23T05:32:57.896Z level=INFO msg="Upload complete. (317ms)"4153lb # [ 58.281090] niks3-server[975]: 2026/09/23 05:32:57 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4154worker1 # [ 58.378076] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04155worker2 # [ 58.293541] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14156worker1 # [ 58.378530] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14157worker2 # [ 58.317562] nix-daemon[839]: reaped child process 914, status = succeeded4158worker2 # [ 58.352839] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584159worker2 # [ 58.373364] nix-grpc-daemon[769]: ts=2026-09-23T05:32:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584160worker2 # [ 58.390983] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.007Z level=INFO msg="All 2 paths already cached"4161client # copying 1 paths to 'lb:50051'...4162client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4163worker2 # [ 58.403781] nix-grpc-daemon[769]: ts=2026-09-23T05:32:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343584164worker2 # [ 58.418079] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4165worker2 # [ 58.609987] nix-daemon[839]: accepted connection from pid 939, user nix-grpc-daemon (trusted)4166worker2 # [ 58.630544] nix-daemon[839]: reaped child process 942, status = succeeded4167lb # [ 58.630218] niks3-server[975]: 2026/09/23 05:32:58 INFO Received uploads request method=POST path=/api/pending_closures4168worker2 # [ 58.649379] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.264Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4169worker2 # [ 58.650887] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.264Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4170lb # [ 58.674536] niks3-server[975]: 2026/09/23 05:32:58 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4171worker2 # [ 58.691953] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.307Z level=INFO msg="Uploading 1 narinfos"4172lb # [ 58.687067] niks3-server[975]: 2026/09/23 05:32:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4173lb # [ 58.698624] niks3-server[975]: 2026/09/23 05:32:58 INFO Signed narinfos id=13 count=14174lb # [ 58.704919] niks3-server[975]: 2026/09/23 05:32:58 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4175lb # [ 58.719645] niks3-server[975]: 2026/09/23 05:32:58 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4176worker2 # [ 58.734794] nix-grpc-daemon[866]: time=2026-09-23T05:32:58.351Z level=INFO msg="Upload complete. (296ms)"4177lb # [ 58.730493] niks3-server[975]: 2026/09/23 05:32:58 INFO Completed upload id=134178lb # [ 58.733107] niks3-server[975]: 2026/09/23 05:32:58 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4179worker2 # [ 58.748299] nix-grpc-daemon[769]: ts=2026-09-23T05:32:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=7524180worker2 # [ 58.760748] nix-grpc-daemon[769]: ts=2026-09-23T05:32:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584181worker2 # [ 58.772970] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4182worker2 # [ 58.785452] nix-daemon[839]: reaped child process 937, status = succeeded4183client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4184client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4185worker2 # [ 58.795151] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4186client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4187lb # [ 59.615864] envoy[904]: conn peer=[2001:db8:1::1]:33886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=774188client # farm-top> LOG-farm-top-sig4189worker2 # [ 59.859124] nix-daemon[839]: reaped child process 946, status = succeeded4190worker2 # [ 59.966642] nix-daemon[839]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4191worker2 # [ 59.982928] nix-daemon[839]: reaped child process 965, status = succeeded4192lb # [ 59.621112] envoy[904]: conn peer=[2001:db8:1::1]:44036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1918 ms=19464193lb # [ 59.985837] niks3-server[975]: 2026/09/23 05:32:59 INFO Received uploads request method=POST path=/api/pending_closures4194worker2 # [ 60.007607] nix-grpc-daemon[866]: time=2026-09-23T05:32:59.621Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4195worker2 # [ 60.009720] nix-grpc-daemon[866]: time=2026-09-23T05:32:59.621Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4196lb # [ 60.028605] niks3-server[975]: 2026/09/23 05:32:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4197worker2 # [ 60.042969] nix-grpc-daemon[866]: time=2026-09-23T05:32:59.659Z level=INFO msg="Uploading 1 narinfos"4198lb # [ 60.041396] niks3-server[975]: 2026/09/23 05:32:59 INFO Signed narinfos id=14 count=14199lb # [ 60.050648] niks3-server[975]: 2026/09/23 05:32:59 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4200lb # [ 60.062569] niks3-server[975]: 2026/09/23 05:32:59 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4201lb # [ 60.074490] niks3-server[975]: 2026/09/23 05:32:59 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4202worker2 # [ 60.089262] nix-grpc-daemon[866]: time=2026-09-23T05:32:59.705Z level=INFO msg="Upload complete. (282ms)"4203lb # [ 60.083841] niks3-server[975]: 2026/09/23 05:32:59 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4204lb # [ 60.087735] niks3-server[975]: 2026/09/23 05:32:59 INFO Completed upload id=144205lb # [ 60.091071] niks3-server[975]: 2026/09/23 05:32:59 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4206worker2 # [ 60.101418] nix-grpc-daemon[769]: ts=2026-09-23T05:32:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14207worker2 # [ 60.121683] nix-daemon[839]: reaped child process 944, status = succeeded4208client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.35 seconds)4209client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4210client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4211client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4212worker1 # [ 60.556330] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:497184213client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4214client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4215client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4216client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4217worker1 # [ 60.594541] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:497184218worker1 # [ 60.603861] nix-grpc-daemon[769]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4219worker1 # [ 60.625973] nix-grpc-daemon[769]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4220worker1 # [ 60.640892] nix-grpc-daemon[769]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4221client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.26 seconds)4222client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4223worker2 # [ 60.787096] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:373404224worker2 # [ 60.829707] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:373404225client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds)4226client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4227client # finding garbage collector roots...4228client # removing stale temporary roots file "/nix/var/nix/temproots/870"4229client # removing stale temporary roots file "/nix/var/nix/temproots/924"4230client # removing stale temporary roots file "/nix/var/nix/temproots/891"4231client # removing stale temporary roots file "/nix/var/nix/temproots/903"4232client # removing stale temporary roots file "/nix/var/nix/temproots/932"4233client # removing stale temporary roots file "/nix/var/nix/temproots/911"4234client # removing stale temporary roots file "/nix/var/nix/temproots/854"4235client # removing stale temporary roots file "/nix/var/nix/temproots/947"4236client # deleting specified paths...4237client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4238client # deleting unused links...4239client # note: hard linking is currently saving 0.0 KiB4240worker1 # [ 61.303593] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:347764241worker1 # [ 61.310539] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764242worker1 # [ 61.324332] nix-grpc-daemon[867]: time=2026-09-23T05:33:00.927Z level=INFO msg="All 1 paths already cached"4243worker1 # [ 61.332515] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347764244worker2 # [ 61.273305] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:343584245client # copying 3 paths...4246worker2 # [ 61.289291] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:343584247client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4248worker2 # [ 61.319432] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:343584249worker2 # [ 61.327347] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_out=2324250worker1 # [ 61.409285] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347764251worker1 # [ 61.417551] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=1764252worker2 # [ 61.362023] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:343584253worker2 # [ 61.368733] nix-grpc-daemon[769]: ts=2026-09-23T05:33:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_out=1284254client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4255client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4256client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.57 seconds)4257(finished: subtest: outputs read back through any worker carry the cache signature, in 7.67 seconds)4258subtest: two clients wanting the same drv build it once4259client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4260client # [ 61.518700] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4261client # Running as unit: dedup1.service; invocation ID: f6d7edc45aaf445c920eb10718d0e6db4262client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4263client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4264worker2 # [ 61.716715] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343584265client # [ 61.717662] nix[999]: don't know how to build these paths:4266client # [ 61.719679] nix[999]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4267worker1 # [ 61.809076] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04268worker1 # [ 61.814514] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14269worker1 # [ 61.856762] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764270worker1 # [ 61.872116] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764271client # [ 61.808258] nix[999]: copying 1 paths to 'lb:50051'...4272worker1 # [ 61.886641] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347764273client # [ 61.812158] nix[999]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4274worker1 # [ 61.903866] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4275worker1 # [ 62.080060] nix-daemon[854]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4276worker1 # [ 62.101928] nix-daemon[854]: reaped child process 1049, status = succeeded4277lb # [ 62.030133] niks3-server[975]: 2026/09/23 05:33:01 INFO Received uploads request method=POST path=/api/pending_closures4278worker1 # [ 62.117660] nix-grpc-daemon[867]: time=2026-09-23T05:33:01.720Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4279worker1 # [ 62.119301] nix-grpc-daemon[867]: time=2026-09-23T05:33:01.720Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4280lb # [ 62.061093] niks3-server[975]: 2026/09/23 05:33:01 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4281worker1 # [ 62.149975] nix-grpc-daemon[867]: time=2026-09-23T05:33:01.752Z level=INFO msg="Uploading 1 narinfos"4282lb # [ 62.076585] niks3-server[975]: 2026/09/23 05:33:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4283lb # [ 62.088558] niks3-server[975]: 2026/09/23 05:33:01 INFO Signed narinfos id=15 count=14284lb # [ 62.095127] niks3-server[975]: 2026/09/23 05:33:01 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4285worker1 # [ 62.183546] nix-grpc-daemon[867]: time=2026-09-23T05:33:01.787Z level=INFO msg="Upload complete. (259ms)"4286lb # [ 62.107151] niks3-server[975]: 2026/09/23 05:33:01 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4287lb # [ 62.115182] niks3-server[975]: 2026/09/23 05:33:01 INFO Completed upload id=154288lb # [ 62.118872] niks3-server[975]: 2026/09/23 05:33:01 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4289worker1 # [ 62.196497] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=4324290worker1 # [ 62.208149] nix-grpc-daemon[769]: ts=2026-09-23T05:33:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764291worker1 # [ 62.219289] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4292worker1 # [ 62.221986] nix-daemon[854]: reaped child process 1044, status = succeeded4293client # [ 62.156302] nix[999]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4294worker1 # [ 62.236723] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4295client # [ 62.614302] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4296client # Running as unit: dedup2.service; invocation ID: 1b8859bce55f4915af7ce096bdd145864297client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4298worker2 # [ 62.851468] nix-grpc-daemon[769]: ts=2026-09-23T05:33:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343584299client # [ 62.852506] nix[1015]: don't know how to build these paths:4300client # [ 62.853513] nix[1015]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4301worker1 # [ 62.941906] nix-grpc-daemon[769]: ts=2026-09-23T05:33:02Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04302client # [ 62.872916] nix[1015]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4303worker1 # [ 62.949437] nix-grpc-daemon[769]: ts=2026-09-23T05:33:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764304client: must succeed: sleep 24305client: (finished: must succeed: sleep 2, in 2.06 seconds)4306client: waiting for success: ! systemctl is-active dedup1 dedup24307worker1 # [ 66.284325] nix-daemon[854]: reaped child process 1053, status = succeeded4308worker1 # [ 66.407898] nix-daemon[854]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)4309worker1 # [ 66.425498] nix-daemon[854]: reaped child process 1088, status = succeeded4310lb # [ 66.356931] niks3-server[975]: 2026/09/23 05:33:05 INFO Received uploads request method=POST path=/api/pending_closures4311worker1 # [ 66.444515] nix-grpc-daemon[867]: time=2026-09-23T05:33:06.047Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4312worker1 # [ 66.446639] nix-grpc-daemon[867]: time=2026-09-23T05:33:06.047Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4313lb # [ 66.408240] niks3-server[975]: 2026/09/23 05:33:06 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4314worker1 # [ 66.494543] nix-grpc-daemon[867]: time=2026-09-23T05:33:06.098Z level=INFO msg="Uploading 1 narinfos"4315lb # [ 66.418563] niks3-server[975]: 2026/09/23 05:33:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4316lb # [ 66.428550] niks3-server[975]: 2026/09/23 05:33:06 INFO Signed narinfos id=16 count=14317lb # [ 66.434648] niks3-server[975]: 2026/09/23 05:33:06 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4318lb # [ 66.442977] niks3-server[975]: 2026/09/23 05:33:06 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4319lb # [ 66.454483] niks3-server[975]: 2026/09/23 05:33:06 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4320worker1 # [ 66.530831] nix-grpc-daemon[867]: time=2026-09-23T05:33:06.134Z level=INFO msg="Upload complete. (278ms)"4321lb # [ 66.461249] niks3-server[975]: 2026/09/23 05:33:06 INFO Completed upload id=164322lb # [ 66.465204] niks3-server[975]: 2026/09/23 05:33:06 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4323worker1 # [ 66.545117] nix-grpc-daemon[769]: ts=2026-09-23T05:33:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14324worker1 # [ 66.564332] nix-grpc-daemon[769]: ts=2026-09-23T05:33:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14325worker1 # [ 66.572231] nix-daemon[854]: reaped child process 1051, status = succeeded4326client # [ 66.527921] systemd[1]: dedup2.service: Deactivated successfully.4327client # [ 66.531226] systemd[1]: dedup2.service: Consumed 204ms CPU time over 3.916s wall clock time, 33.2M memory peak, 6.1K incoming IP traffic, 5.8K outgoing IP traffic.4328client # [ 66.539427] systemd[1]: dedup1.service: Deactivated successfully.4329client # [ 66.543000] systemd[1]: dedup1.service: Consumed 178ms CPU time over 5.021s wall clock time, 41.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4330client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4331client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4332client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4333worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4334worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4335worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4336worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4337worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4338worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4339worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4340worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4341(finished: subtest: two clients wanting the same drv build it once, in 6.12 seconds)4342subtest: --timeout and --max-silent-time reach the builder4343worker1 # [ 67.888922] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:347764344worker1 # [ 67.905797] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04345worker1 # [ 67.911552] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14346worker2 # [ 67.885772] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584347worker2 # [ 67.901715] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584348worker2 # [ 67.916791] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343584349worker2 # [ 67.931644] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4350worker2 # [ 68.092157] nix-daemon[839]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4351worker2 # [ 68.111913] nix-daemon[839]: reaped child process 993, status = succeeded4352lb # [ 68.110347] niks3-server[975]: 2026/09/23 05:33:07 INFO Received uploads request method=POST path=/api/pending_closures4353worker2 # [ 68.126649] nix-grpc-daemon[866]: time=2026-09-23T05:33:07.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4354worker2 # [ 68.128431] nix-grpc-daemon[866]: time=2026-09-23T05:33:07.742Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4355lb # [ 68.138936] niks3-server[975]: 2026/09/23 05:33:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4356worker2 # [ 68.154099] nix-grpc-daemon[866]: time=2026-09-23T05:33:07.770Z level=INFO msg="Uploading 1 narinfos"4357lb # [ 68.153765] niks3-server[975]: 2026/09/23 05:33:07 INFO Signed narinfos id=17 count=14358lb # [ 68.163535] niks3-server[975]: 2026/09/23 05:33:07 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4359lb # [ 68.173981] niks3-server[975]: 2026/09/23 05:33:07 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4360worker2 # [ 68.194332] nix-grpc-daemon[866]: time=2026-09-23T05:33:07.810Z level=INFO msg="Upload complete. (245ms)"4361lb # [ 68.188940] niks3-server[975]: 2026/09/23 05:33:07 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4362lb # [ 68.196691] niks3-server[975]: 2026/09/23 05:33:07 INFO Completed upload id=174363worker2 # [ 68.204431] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=4244364lb # [ 68.199351] niks3-server[975]: 2026/09/23 05:33:07 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4365worker2 # [ 68.220607] nix-grpc-daemon[769]: ts=2026-09-23T05:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584366worker2 # [ 68.229885] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4367worker2 # [ 68.233115] nix-daemon[839]: reaped child process 988, status = succeeded4368worker2 # [ 68.245916] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4369lb # [ 69.616370] envoy[904]: conn peer=[2001:db8:1::1]:44052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9554 tx=4401 ms=60994370lb # [ 69.616703] envoy[904]: conn peer=[2001:db8:1::1]:59522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=1294371lb # [ 69.616810] envoy[904]: conn peer=[2001:db8:1::1]:59534 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=1114372lb # [ 69.616925] envoy[904]: conn peer=[2001:db8:1::1]:59538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=924373lb # [ 69.617022] envoy[904]: conn peer=[2001:db8:1::1]:59506 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2891 ms=2184374lb # [ 69.617161] envoy[904]: conn peer=[2001:db8:1::1]:59558 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=36424375lb # [ 69.617261] envoy[904]: conn peer=[2001:db8:1::1]:59544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1563 ms=47884376worker2 # [ 74.065269] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04377worker2 # [ 74.103562] nix-daemon[839]: reaped child process 995, status = succeeded4378worker2 # [ 74.106417] nix-daemon[839]: reaped child process 997, status = succeeded4379worker1 # [ 74.369051] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:347764380worker1 # [ 74.383429] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04381worker1 # [ 74.390703] nix-grpc-daemon[769]: ts=2026-09-23T05:33:13Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14382worker1 # [ 74.399936] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764383worker1 # [ 74.412966] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764384worker1 # [ 74.427582] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347764385worker1 # [ 74.443475] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4386worker1 # [ 74.596935] nix-daemon[854]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4387worker1 # [ 74.618604] nix-daemon[854]: reaped child process 1108, status = succeeded4388lb # [ 74.541402] niks3-server[975]: 2026/09/23 05:33:14 INFO Received uploads request method=POST path=/api/pending_closures4389worker1 # [ 74.628528] nix-grpc-daemon[867]: time=2026-09-23T05:33:14.231Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4390worker1 # [ 74.630381] nix-grpc-daemon[867]: time=2026-09-23T05:33:14.231Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4391lb # [ 74.572634] niks3-server[975]: 2026/09/23 05:33:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4392worker1 # [ 74.659978] nix-grpc-daemon[867]: time=2026-09-23T05:33:14.263Z level=INFO msg="Uploading 1 narinfos"4393lb # [ 74.584361] niks3-server[975]: 2026/09/23 05:33:14 INFO Signed narinfos id=18 count=14394lb # [ 74.597848] niks3-server[975]: 2026/09/23 05:33:14 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4395lb # [ 74.607200] niks3-server[975]: 2026/09/23 05:33:14 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4396lb # [ 74.618335] niks3-server[975]: 2026/09/23 05:33:14 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4397worker1 # [ 74.692788] nix-grpc-daemon[867]: time=2026-09-23T05:33:14.296Z level=INFO msg="Upload complete. (233ms)"4398lb # [ 74.625765] niks3-server[975]: 2026/09/23 05:33:14 INFO Completed upload id=184399lb # [ 74.628256] niks3-server[975]: 2026/09/23 05:33:14 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4400worker1 # [ 74.703182] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=4244401worker1 # [ 74.720696] nix-grpc-daemon[769]: ts=2026-09-23T05:33:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764402worker1 # [ 74.724278] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4403worker1 # [ 74.729697] nix-daemon[854]: reaped child process 1103, status = succeeded4404worker1 # [ 74.741025] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4405lb # [ 79.616962] envoy[904]: conn peer=[2001:db8:1::1]:59574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=953 ms=62654406worker1 # [ 81.157383] nix-grpc-daemon[769]: ts=2026-09-23T05:33:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04407worker1 # [ 81.197220] nix-daemon[854]: reaped child process 1110, status = succeeded4408worker1 # [ 81.199988] nix-daemon[854]: reaped child process 1112, status = succeeded4409(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.67 seconds)4410subtest: the first client leaving does not fail the second4411client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4412client # [ 81.321211] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4413client # Running as unit: share1.service; invocation ID: 9425ec8079084282bf7b2922f1ba8cb54414client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4415client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4416client # [ 81.530884] nix[1089]: don't know how to build these paths:4417client # [ 81.531847] nix[1089]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4418worker2 # [ 81.537452] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343584419worker1 # [ 81.620136] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04420worker1 # [ 81.626515] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14421worker2 # [ 81.559089] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584422worker2 # [ 81.575299] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584423client # [ 81.579969] nix[1089]: copying 1 paths to 'lb:50051'...4424client # [ 81.582496] nix[1089]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4425worker2 # [ 81.591168] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343584426worker2 # [ 81.606665] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4427worker2 # [ 81.769329] nix-daemon[839]: accepted connection from pid 1020, user nix-grpc-daemon (trusted)4428worker2 # [ 81.787734] nix-daemon[839]: reaped child process 1023, status = succeeded4429lb # [ 81.789781] niks3-server[975]: 2026/09/23 05:33:21 INFO Received uploads request method=POST path=/api/pending_closures4430worker2 # [ 81.806624] nix-grpc-daemon[866]: time=2026-09-23T05:33:21.422Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4431worker2 # [ 81.808409] nix-grpc-daemon[866]: time=2026-09-23T05:33:21.422Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4432lb # [ 81.818742] niks3-server[975]: 2026/09/23 05:33:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4433worker2 # [ 81.834735] nix-grpc-daemon[866]: time=2026-09-23T05:33:21.450Z level=INFO msg="Uploading 1 narinfos"4434lb # [ 81.831453] niks3-server[975]: 2026/09/23 05:33:21 INFO Signed narinfos id=19 count=14435lb # [ 81.841225] niks3-server[975]: 2026/09/23 05:33:21 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4436lb # [ 81.850606] niks3-server[975]: 2026/09/23 05:33:21 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4437lb # [ 81.861861] niks3-server[975]: 2026/09/23 05:33:21 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4438worker2 # [ 81.874301] nix-grpc-daemon[866]: time=2026-09-23T05:33:21.490Z level=INFO msg="Upload complete. (250ms)"4439lb # [ 81.870408] niks3-server[975]: 2026/09/23 05:33:21 INFO Completed upload id=194440lb # [ 81.874295] niks3-server[975]: 2026/09/23 05:33:21 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4441worker2 # [ 81.884424] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=4324442worker2 # [ 81.903919] nix-grpc-daemon[769]: ts=2026-09-23T05:33:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584443worker2 # [ 81.911266] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4444worker2 # [ 81.917405] nix-daemon[839]: reaped child process 1018, status = succeeded4445worker2 # [ 81.930438] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4446client # [ 81.921957] nix[1089]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4447client # [ 82.427177] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4448client # Running as unit: share2.service; invocation ID: 551b97cc432a4961834d3078e21d79424449client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4450worker1 # [ 82.693668] nix-grpc-daemon[769]: ts=2026-09-23T05:33:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:347764451client # [ 82.623864] nix[1105]: don't know how to build these paths:4452client # [ 82.625586] nix[1105]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4453worker1 # [ 82.710593] nix-grpc-daemon[769]: ts=2026-09-23T05:33:22Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04454worker2 # [ 82.645278] nix-grpc-daemon[769]: ts=2026-09-23T05:33:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584455client # [ 82.644996] nix[1105]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4456client: must succeed: sleep 2; systemctl kill -s INT share14457client # [ 85.762221] systemd[1]: share1.service: Sent signal SIGINT to main process 1089 (nix) on client request.4458client # [ 85.772956] nix[1089]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4459client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4460client: waiting for success: ! systemctl is-active share14461client # [ 85.788595] nix[1089]: error: interrupted by the user4462client # [ 85.843522] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4463client # [ 85.845844] systemd[1]: share1.service: Failed with result 'exit-code'.4464client # [ 85.849177] systemd[1]: share1.service: Consumed 174ms CPU time over 4.527s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8K outgoing IP traffic.4465client: (finished: waiting for success: ! systemctl is-active share1, in 1.11 seconds)4466client: waiting for success: ! systemctl is-active share24467worker2 # [ 87.397996] nix-daemon[839]: reaped child process 1027, status = succeeded4468worker2 # [ 87.535238] nix-daemon[839]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)4469worker2 # [ 87.552777] nix-daemon[839]: reaped child process 1064, status = succeeded4470lb # [ 87.554412] niks3-server[975]: 2026/09/23 05:33:27 INFO Received uploads request method=POST path=/api/pending_closures4471worker2 # [ 87.572454] nix-grpc-daemon[866]: time=2026-09-23T05:33:27.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4472worker2 # [ 87.574188] nix-grpc-daemon[866]: time=2026-09-23T05:33:27.187Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4473lb # [ 87.593523] niks3-server[975]: 2026/09/23 05:33:27 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4474lb # [ 87.608910] niks3-server[975]: 2026/09/23 05:33:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4475worker2 # [ 87.621771] nix-grpc-daemon[866]: time=2026-09-23T05:33:27.237Z level=INFO msg="Uploading 1 narinfos"4476lb # [ 87.618144] niks3-server[975]: 2026/09/23 05:33:27 INFO Signed narinfos id=20 count=14477lb # [ 87.623181] niks3-server[975]: 2026/09/23 05:33:27 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4478lb # [ 87.633215] niks3-server[975]: 2026/09/23 05:33:27 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4479worker2 # [ 87.645242] nix-grpc-daemon[866]: time=2026-09-23T05:33:27.261Z level=INFO msg="Upload complete. (294ms)"4480lb # [ 87.639440] niks3-server[975]: 2026/09/23 05:33:27 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4481lb # [ 87.643540] niks3-server[975]: 2026/09/23 05:33:27 INFO Completed upload id=204482lb # [ 87.646474] niks3-server[975]: 2026/09/23 05:33:27 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4483worker2 # [ 87.657219] nix-grpc-daemon[769]: ts=2026-09-23T05:33:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14484worker2 # [ 87.676424] nix-grpc-daemon[769]: ts=2026-09-23T05:33:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14485worker2 # [ 87.684251] nix-daemon[839]: reaped child process 1025, status = succeeded4486client # [ 87.686914] systemd[1]: share2.service: Deactivated successfully.4487client # [ 87.691907] systemd[1]: share2.service: Consumed 170ms CPU time over 5.261s wall clock time, 33.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4488client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4489client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4490client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4491(finished: subtest: the first client leaving does not fail the second, in 7.06 seconds)4492subtest: build hook: nix-daemon with builders = grpc://lb4493client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4494client # [ 88.451439] systemd[1]: Started Nix Daemon.4495client # [ 88.565587] nix-daemon[1157]: accepted connection from pid 1154, user root (trusted)4496worker2 # [ 88.717173] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:343584497worker1 # [ 88.791982] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764498worker1 # [ 88.806761] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04499worker1 # [ 88.814018] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14500worker1 # [ 88.823139] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764501worker1 # [ 88.833954] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764502worker1 # [ 88.847977] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347764503worker1 # [ 88.862791] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4504worker1 # [ 89.013971] nix-daemon[854]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4505worker1 # [ 89.031794] nix-daemon[854]: reaped child process 1160, status = succeeded4506lb # [ 88.962315] niks3-server[975]: 2026/09/23 05:33:28 INFO Received uploads request method=POST path=/api/pending_closures4507worker1 # [ 89.048572] nix-grpc-daemon[867]: time=2026-09-23T05:33:28.652Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4508worker1 # [ 89.050801] nix-grpc-daemon[867]: time=2026-09-23T05:33:28.654Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4509lb # [ 88.993245] niks3-server[975]: 2026/09/23 05:33:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4510worker1 # [ 89.079154] nix-grpc-daemon[867]: time=2026-09-23T05:33:28.682Z level=INFO msg="Uploading 1 narinfos"4511lb # [ 89.007826] niks3-server[975]: 2026/09/23 05:33:28 INFO Signed narinfos id=21 count=14512lb # [ 89.016510] niks3-server[975]: 2026/09/23 05:33:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4513lb # [ 89.025479] niks3-server[975]: 2026/09/23 05:33:28 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4514lb # [ 89.034780] niks3-server[975]: 2026/09/23 05:33:28 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4515lb # [ 89.041895] niks3-server[975]: 2026/09/23 05:33:28 INFO Completed upload id=214516worker1 # [ 89.118750] nix-grpc-daemon[867]: time=2026-09-23T05:33:28.722Z level=INFO msg="Upload complete. (240ms)"4517lb # [ 89.045987] niks3-server[975]: 2026/09/23 05:33:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4518worker1 # [ 89.128312] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=5124519worker1 # [ 89.147008] nix-grpc-daemon[769]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764520worker1 # [ 89.156389] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4521worker1 # [ 89.159977] nix-daemon[854]: reaped child process 1155, status = succeeded4522worker1 # [ 89.173375] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4523lb # [ 89.621157] envoy[904]: conn peer=[2001:db8:1::1]:47364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=934 ms=68144524lb # [ 89.626106] envoy[904]: conn peer=[2001:db8:1::1]:43496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=797 ms=42634525lb # [ 89.630864] envoy[904]: conn peer=[2001:db8:1::1]:43510 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=50504526worker1 # [ 90.664784] nix-daemon[854]: reaped child process 1164, status = succeeded4527worker1 # [ 90.747466] nix-daemon[854]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)4528worker1 # [ 90.765528] nix-daemon[854]: reaped child process 1183, status = succeeded4529lb # [ 90.694662] niks3-server[975]: 2026/09/23 05:33:30 INFO Received uploads request method=POST path=/api/pending_closures4530worker1 # [ 90.783715] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.385Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4531worker1 # [ 90.785477] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.385Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4532lb # [ 90.739081] niks3-server[975]: 2026/09/23 05:33:30 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4533worker1 # [ 90.828687] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.432Z level=INFO msg="Uploading 1 narinfos"4534lb # [ 90.754224] niks3-server[975]: 2026/09/23 05:33:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4535lb # [ 90.763180] niks3-server[975]: 2026/09/23 05:33:30 INFO Signed narinfos id=22 count=14536lb # [ 90.770501] niks3-server[975]: 2026/09/23 05:33:30 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4537worker1 # [ 90.853721] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.457Z level=INFO msg="Upload complete. (269ms)"4538lb # [ 90.780268] niks3-server[975]: 2026/09/23 05:33:30 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4539lb # [ 90.788242] niks3-server[975]: 2026/09/23 05:33:30 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4540lb # [ 90.793192] niks3-server[975]: 2026/09/23 05:33:30 INFO Completed upload id=224541worker1 # [ 90.866405] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14542lb # [ 90.797654] niks3-server[975]: 2026/09/23 05:33:30 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4543worker1 # [ 90.885257] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347764544worker1 # [ 90.890693] nix-daemon[854]: reaped child process 1162, status = succeeded4545worker1 # [ 90.912158] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347764546worker1 # [ 90.919179] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=1284547worker2 # [ 91.125091] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:343584548worker1 # [ 91.198638] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764549worker1 # [ 91.210922] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.814Z level=INFO msg="All 1 paths already cached"4550worker1 # [ 91.229178] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04551worker1 # [ 91.235548] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14552worker1 # [ 91.277327] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764553worker1 # [ 91.292565] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764554worker1 # [ 91.307562] nix-grpc-daemon[867]: time=2026-09-23T05:33:30.911Z level=INFO msg="All 1 paths already cached"4555worker1 # [ 91.317307] nix-grpc-daemon[769]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347764556worker1 # [ 91.334353] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4557worker1 # [ 91.486473] nix-daemon[854]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)4558worker1 # [ 91.505349] nix-daemon[854]: reaped child process 1192, status = succeeded4559lb # [ 91.433677] niks3-server[975]: 2026/09/23 05:33:31 INFO Received uploads request method=POST path=/api/pending_closures4560worker1 # [ 91.520710] nix-grpc-daemon[867]: time=2026-09-23T05:33:31.124Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4561worker1 # [ 91.522272] nix-grpc-daemon[867]: time=2026-09-23T05:33:31.124Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4562lb # [ 91.464390] niks3-server[975]: 2026/09/23 05:33:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4563worker1 # [ 91.548085] nix-grpc-daemon[867]: time=2026-09-23T05:33:31.151Z level=INFO msg="Uploading 1 narinfos"4564lb # [ 91.479717] niks3-server[975]: 2026/09/23 05:33:31 INFO Signed narinfos id=23 count=14565lb # [ 91.487361] niks3-server[975]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4566lb # [ 91.494980] niks3-server[975]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4567worker1 # [ 91.583662] nix-grpc-daemon[867]: time=2026-09-23T05:33:31.187Z level=INFO msg="Upload complete. (235ms)"4568lb # [ 91.507784] niks3-server[975]: 2026/09/23 05:33:31 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4569lb # [ 91.514918] niks3-server[975]: 2026/09/23 05:33:31 INFO Completed upload id=234570worker1 # [ 91.592285] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=6244571lb # [ 91.518326] niks3-server[975]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4572worker1 # [ 91.604560] nix-grpc-daemon[769]: ts=2026-09-23T05:33:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764573worker1 # [ 91.616894] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4574worker1 # [ 91.622286] nix-daemon[854]: reaped child process 1187, status = succeeded4575worker1 # [ 91.634575] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4576worker1 # [ 92.327663] nix-daemon[854]: reaped child process 1196, status = succeeded4577worker1 # [ 92.416402] nix-daemon[854]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)4578worker1 # [ 92.435779] nix-daemon[854]: reaped child process 1215, status = succeeded4579lb # [ 92.363594] niks3-server[975]: 2026/09/23 05:33:31 INFO Received uploads request method=POST path=/api/pending_closures4580worker1 # [ 92.452566] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.055Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4581worker1 # [ 92.454685] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.055Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4582worker1 # [ 92.510727] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.114Z level=INFO msg="Uploading 1 narinfos"4583lb # [ 92.421566] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4584lb # [ 92.439821] niks3-server[975]: 2026/09/23 05:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4585lb # [ 92.449777] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4586lb # [ 92.458150] niks3-server[975]: 2026/09/23 05:33:32 INFO Signed narinfos id=24 count=14587worker1 # [ 92.543325] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.147Z level=INFO msg="Upload complete. (296ms)"4588lb # [ 92.468194] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4589lb # [ 92.473403] niks3-server[975]: 2026/09/23 05:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4590lb # [ 92.479369] niks3-server[975]: 2026/09/23 05:33:32 INFO Completed upload id=244591worker1 # [ 92.557340] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14592lb # [ 92.485717] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4593worker2 # [ 92.506439] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:343584594worker1 # [ 92.580319] nix-daemon[854]: reaped child process 1194, status = succeeded4595worker2 # [ 92.528338] nix-grpc-daemon[769]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4596worker2 # [ 92.545765] nix-grpc-daemon[769]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4597worker1 # [ 92.657111] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347764598worker1 # [ 92.665680] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=1764599worker1 # [ 92.905057] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:347764600worker1 # [ 92.912894] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764601worker1 # [ 92.920863] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.522Z level=INFO msg="All 2 paths already cached"4602worker1 # [ 92.933911] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04603worker1 # [ 92.941241] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14604worker1 # [ 92.950668] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764605worker1 # [ 92.962697] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764606worker1 # [ 92.977817] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.581Z level=INFO msg="All 2 paths already cached"4607worker1 # [ 92.988189] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347764608worker1 # [ 93.002914] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4609worker1 # [ 93.157832] nix-daemon[854]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)4610worker1 # [ 93.176289] nix-daemon[854]: reaped child process 1224, status = succeeded4611lb # [ 93.105937] niks3-server[975]: 2026/09/23 05:33:32 INFO Received uploads request method=POST path=/api/pending_closures4612worker1 # [ 93.192759] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.796Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4613worker1 # [ 93.194350] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.796Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4614lb # [ 93.140562] niks3-server[975]: 2026/09/23 05:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4615worker1 # [ 93.227559] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.831Z level=INFO msg="Uploading 1 narinfos"4616lb # [ 93.156191] niks3-server[975]: 2026/09/23 05:33:32 INFO Signed narinfos id=25 count=14617lb # [ 93.159102] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4618lb # [ 93.172154] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4619lb # [ 93.184944] niks3-server[975]: 2026/09/23 05:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4620lb # [ 93.191796] niks3-server[975]: 2026/09/23 05:33:32 INFO Completed upload id=254621worker1 # [ 93.267694] nix-grpc-daemon[867]: time=2026-09-23T05:33:32.871Z level=INFO msg="Upload complete. (249ms)"4622lb # [ 93.194694] niks3-server[975]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4623worker1 # [ 93.279521] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=7524624worker1 # [ 93.296551] nix-grpc-daemon[769]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764625worker1 # [ 93.306886] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4626worker1 # [ 93.311163] nix-daemon[854]: reaped child process 1219, status = succeeded4627worker1 # [ 93.327001] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4628worker1 # [ 94.230503] nix-daemon[854]: reaped child process 1228, status = succeeded4629worker1 # [ 94.299588] nix-daemon[854]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)4630worker1 # [ 94.320922] nix-daemon[854]: reaped child process 1247, status = succeeded4631lb # [ 94.250521] niks3-server[975]: 2026/09/23 05:33:33 INFO Received uploads request method=POST path=/api/pending_closures4632worker1 # [ 94.340402] nix-grpc-daemon[867]: time=2026-09-23T05:33:33.941Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4633worker1 # [ 94.342012] nix-grpc-daemon[867]: time=2026-09-23T05:33:33.941Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4634lb # [ 94.297909] niks3-server[975]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4635worker1 # [ 94.385816] nix-grpc-daemon[867]: time=2026-09-23T05:33:33.989Z level=INFO msg="Uploading 1 narinfos"4636lb # [ 94.311614] niks3-server[975]: 2026/09/23 05:33:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4637lb # [ 94.321207] niks3-server[975]: 2026/09/23 05:33:33 INFO Signed narinfos id=26 count=14638lb # [ 94.323601] niks3-server[975]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4639lb # [ 94.331608] niks3-server[975]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4640worker1 # [ 94.419826] nix-grpc-daemon[867]: time=2026-09-23T05:33:34.023Z level=INFO msg="Upload complete. (281ms)"4641lb # [ 94.344319] niks3-server[975]: 2026/09/23 05:33:33 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4642lb # [ 94.349087] niks3-server[975]: 2026/09/23 05:33:33 INFO Completed upload id=264643lb # [ 94.352785] niks3-server[975]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4644worker1 # [ 94.433377] nix-grpc-daemon[769]: ts=2026-09-23T05:33:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14645worker1 # [ 94.457679] nix-grpc-daemon[769]: ts=2026-09-23T05:33:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347764646worker1 # [ 94.462869] nix-daemon[854]: reaped child process 1226, status = succeeded4647worker1 # [ 94.479535] nix-grpc-daemon[769]: ts=2026-09-23T05:33:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347764648worker1 # [ 94.486935] nix-grpc-daemon[769]: ts=2026-09-23T05:33:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=2324649client # [ 94.545215] nix-daemon[1157]: reaped child process 1159, status = succeeded4650client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.25 seconds)4651client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4652client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4653client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4654client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4655client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4656client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4657(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.37 seconds)4658subtest: an input only one worker has still reaches the builder4659client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4660client # this derivation will be built:4661client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4662client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4663client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.31 seconds)4664client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4665client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4666worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4667worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)4668worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4669worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4670client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24671client # [ 95.524417] nix-daemon[1157]: accepted connection from pid 1238, user root (trusted)4672client # this derivation will be built:4673client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4674client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4675worker1 # [ 95.755490] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:347764676worker2 # [ 95.690742] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584677client # copying 1 paths...4678client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4679worker2 # [ 95.706334] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343584680worker2 # [ 95.721395] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4681worker2 # [ 95.894042] nix-daemon[839]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)4682worker2 # [ 95.914017] nix-daemon[839]: reaped child process 1076, status = succeeded4683lb # [ 95.915057] niks3-server[975]: 2026/09/23 05:33:35 INFO Received uploads request method=POST path=/api/pending_closures4684worker2 # [ 95.932778] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.548Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4685worker2 # [ 95.934401] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.548Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4686lb # [ 95.949114] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4687worker2 # [ 95.964634] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.580Z level=INFO msg="Uploading 1 narinfos"4688lb # [ 95.961451] niks3-server[975]: 2026/09/23 05:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4689lb # [ 95.970324] niks3-server[975]: 2026/09/23 05:33:35 INFO Signed narinfos id=27 count=14690worker2 # [ 95.987221] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.603Z level=INFO msg="Upload complete. (249ms)"4691lb # [ 95.980553] niks3-server[975]: 2026/09/23 05:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4692lb # [ 95.988502] niks3-server[975]: 2026/09/23 05:33:35 INFO Completed upload id=274693lb # [ 95.991489] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4694lb # [ 95.998647] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4695worker2 # [ 95.996360] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=1204696worker2 # [ 96.011359] nix-daemon[839]: reaped child process 1071, status = succeeded4697worker1 # [ 96.083309] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04698worker1 # [ 96.089841] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14699worker2 # [ 96.058664] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584700worker2 # [ 96.074275] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584701client # copying 2 paths to 'lb:50051'...4702client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4703worker2 # [ 96.092579] nix-grpc-daemon[769]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343584704client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4705worker2 # [ 96.106064] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4706worker2 # [ 96.255988] nix-daemon[839]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)4707worker2 # [ 96.273344] nix-daemon[839]: reaped child process 1083, status = succeeded4708lb # [ 96.272991] niks3-server[975]: 2026/09/23 05:33:35 INFO Received uploads request method=POST path=/api/pending_closures4709lb # [ 96.282470] niks3-server[975]: 2026/09/23 05:33:35 INFO Received uploads request method=POST path=/api/pending_closures4710worker2 # [ 96.299434] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.914Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4711worker2 # [ 96.302387] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.914Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4712worker2 # [ 96.305066] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.914Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4713lb # [ 96.329112] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4714lb # [ 96.342366] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4715lb # [ 96.354377] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4716worker2 # [ 96.360852] nix-grpc-daemon[866]: time=2026-09-23T05:33:35.976Z level=INFO msg="Uploading 3 narinfos"4717lb # [ 96.361553] niks3-server[975]: 2026/09/23 05:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4718lb # [ 96.368069] niks3-server[975]: 2026/09/23 05:33:35 INFO Signed narinfos id=28 count=14719lb # [ 96.372512] niks3-server[975]: 2026/09/23 05:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4720lb # [ 96.379736] niks3-server[975]: 2026/09/23 05:33:35 INFO Signed narinfos id=29 count=24721lb # [ 96.385618] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4722lb # [ 96.393028] niks3-server[975]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4723lb # [ 96.399824] niks3-server[975]: 2026/09/23 05:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4724lb # [ 96.407769] niks3-server[975]: 2026/09/23 05:33:36 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4725lb # [ 96.412366] niks3-server[975]: 2026/09/23 05:33:36 INFO Completed upload id=284726lb # [ 96.414297] niks3-server[975]: 2026/09/23 05:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4727lb # [ 96.417301] niks3-server[975]: 2026/09/23 05:33:36 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4728lb # [ 96.419520] niks3-server[975]: 2026/09/23 05:33:36 INFO Completed upload id=294729worker2 # [ 96.423955] nix-grpc-daemon[866]: time=2026-09-23T05:33:36.040Z level=INFO msg="Upload complete. (300ms)"4730worker2 # [ 96.436254] nix-grpc-daemon[769]: ts=2026-09-23T05:33:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=2 nar_bytes_in=9844731worker2 # [ 96.452624] nix-grpc-daemon[769]: ts=2026-09-23T05:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584732worker2 # [ 96.463874] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4733worker2 # [ 96.469250] nix-daemon[839]: reaped child process 1078, status = succeeded4734client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4735worker2 # [ 96.484262] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4736client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4737client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4738worker2 # [ 97.749739] nix-daemon[839]: reaped child process 1087, status = succeeded4739worker2 # [ 97.889024] nix-daemon[839]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)4740worker2 # [ 97.907249] nix-daemon[839]: reaped child process 1106, status = succeeded4741lb # [ 97.907029] niks3-server[975]: 2026/09/23 05:33:37 INFO Received uploads request method=POST path=/api/pending_closures4742worker2 # [ 97.925637] nix-grpc-daemon[866]: time=2026-09-23T05:33:37.540Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4743lb # [ 97.939454] niks3-server[975]: 2026/09/23 05:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign4744worker2 # [ 97.955808] nix-grpc-daemon[866]: time=2026-09-23T05:33:37.572Z level=INFO msg="Uploading 1 narinfos"4745lb # [ 97.954493] niks3-server[975]: 2026/09/23 05:33:37 INFO Signed narinfos id=30 count=14746lb # [ 97.961542] niks3-server[975]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4747worker2 # [ 97.980823] nix-grpc-daemon[866]: time=2026-09-23T05:33:37.597Z level=INFO msg="Upload complete. (280ms)"4748lb # [ 97.969095] niks3-server[975]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4749lb # [ 97.982901] niks3-server[975]: 2026/09/23 05:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete4750lb # [ 97.986677] niks3-server[975]: 2026/09/23 05:33:37 INFO Completed upload id=304751worker2 # [ 97.993027] nix-grpc-daemon[769]: ts=2026-09-23T05:33:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=14752lb # [ 97.989286] niks3-server[975]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4753worker2 # [ 98.008147] nix-daemon[839]: reaped child process 1085, status = succeeded4754client # copying 1 paths...4755worker2 # [ 98.036537] nix-grpc-daemon[769]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:343584756client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4757worker2 # [ 98.069348] nix-grpc-daemon[769]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:343584758worker2 # [ 98.076164] nix-grpc-daemon[769]: ts=2026-09-23T05:33:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_out=1204759client # [ 98.198742] nix-daemon[1157]: reaped child process 1241, status = succeeded4760client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.86 seconds)4761client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24762client # [ 98.374446] nix-daemon[1157]: accepted connection from pid 1259, user root (trusted)4763client # this derivation will be built:4764client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4765client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4766worker1 # [ 98.597222] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:347764767worker2 # [ 98.529644] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584768worker2 # [ 98.539293] nix-grpc-daemon[866]: time=2026-09-23T05:33:38.155Z level=INFO msg="All 1 paths already cached"4769worker1 # [ 98.626506] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04770worker1 # [ 98.636310] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14771worker1 # [ 98.645334] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764772worker1 # [ 98.654036] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347764773client # copying 1 paths to 'lb:50051'...4774client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4775worker1 # [ 98.666936] nix-grpc-daemon[769]: ts=2026-09-23T05:33:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347764776worker1 # [ 98.681922] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4777worker1 # [ 98.702015] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5)4778worker1 # [ 98.777340] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 2/5)4779worker1 # [ 98.811028] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 3/5)4780worker1 # [ 98.852938] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 320 ms (attempt 4/5)4781worker1 # [ 99.175501] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4782worker1 # [ 99.182271] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4783worker1 # [ 99.190841] nix-grpc-daemon[769]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4784worker1 # [ 99.364096] nix-daemon[854]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)4785worker1 # [ 99.382518] nix-daemon[854]: reaped child process 1268, status = succeeded4786lb # [ 99.314381] niks3-server[975]: 2026/09/23 05:33:38 INFO Received uploads request method=POST path=/api/pending_closures4787worker1 # [ 99.401419] nix-grpc-daemon[867]: time=2026-09-23T05:33:39.004Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4788worker1 # [ 99.402994] nix-grpc-daemon[867]: time=2026-09-23T05:33:39.004Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4789lb # [ 99.344579] niks3-server[975]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4790worker1 # [ 99.433375] nix-grpc-daemon[867]: time=2026-09-23T05:33:39.037Z level=INFO msg="Uploading 1 narinfos"4791lb # [ 99.359968] niks3-server[975]: 2026/09/23 05:33:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign4792lb # [ 99.368968] niks3-server[975]: 2026/09/23 05:33:38 INFO Signed narinfos id=31 count=14793lb # [ 99.378035] niks3-server[975]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4794worker1 # [ 99.460211] nix-grpc-daemon[867]: time=2026-09-23T05:33:39.064Z level=INFO msg="Upload complete. (250ms)"4795lb # [ 99.383927] niks3-server[975]: 2026/09/23 05:33:38 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete4796lb # [ 99.391388] niks3-server[975]: 2026/09/23 05:33:38 INFO Completed upload id=314797worker1 # [ 99.468143] nix-grpc-daemon[769]: ts=2026-09-23T05:33:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_in=5684798lb # [ 99.396311] niks3-server[975]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4799worker1 # [ 99.479951] nix-grpc-daemon[769]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347764800worker1 # [ 99.492518] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4801worker1 # [ 99.497614] nix-daemon[854]: reaped child process 1258, status = succeeded4802worker1 # [ 99.508861] nix-daemon[854]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4803client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4804client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4805lb # [ 99.619512] envoy[904]: conn peer=[2001:db8:1::1]:43522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3440 tx=1966 ms=21914806lb # [ 99.624605] envoy[904]: conn peer=[2001:db8:1::1]:37418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=684807lb # [ 99.629228] envoy[904]: conn peer=[2001:db8:1::1]:37426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3886 tx=2155 ms=14874808lb # [ 99.633952] envoy[904]: conn peer=[2001:db8:1::1]:37438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=294809lb # [ 99.638614] envoy[904]: conn peer=[2001:db8:1::1]:37440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4338 tx=2389 ms=16414810lb # [ 99.643528] envoy[904]: conn peer=[2001:db8:1::1]:37454 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=764811lb # [ 99.648316] envoy[904]: conn peer=[2001:db8:1::1]:37460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4271 tx=2072 ms=24504812lb # [ 99.653009] envoy[904]: conn peer=[2001:db8:1::1]:37466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=704813client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4814worker1 # [ 100.619099] nix-daemon[854]: reaped child process 1272, status = succeeded4815worker1 # [ 100.690200] nix-daemon[854]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)4816worker1 # [ 100.706523] nix-daemon[854]: reaped child process 1291, status = succeeded4817lb # [ 100.637044] niks3-server[975]: 2026/09/23 05:33:40 INFO Received uploads request method=POST path=/api/pending_closures4818worker1 # [ 100.724662] nix-grpc-daemon[867]: time=2026-09-23T05:33:40.326Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4819lb # [ 100.663453] niks3-server[975]: 2026/09/23 05:33:40 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4820lb # [ 100.678202] niks3-server[975]: 2026/09/23 05:33:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign4821worker1 # [ 100.761550] nix-grpc-daemon[867]: time=2026-09-23T05:33:40.365Z level=INFO msg="Uploading 1 narinfos"4822lb # [ 100.690952] niks3-server[975]: 2026/09/23 05:33:40 INFO Signed narinfos id=32 count=14823lb # [ 100.698979] niks3-server[975]: 2026/09/23 05:33:40 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4824worker1 # [ 100.788456] nix-grpc-daemon[867]: time=2026-09-23T05:33:40.392Z level=INFO msg="Upload complete. (265ms)"4825lb # [ 100.711616] niks3-server[975]: 2026/09/23 05:33:40 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete4826lb # [ 100.717127] niks3-server[975]: 2026/09/23 05:33:40 INFO Completed upload id=324827lb # [ 100.723616] niks3-server[975]: 2026/09/23 05:33:40 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4828worker1 # [ 100.799640] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=14829client # copying 1 paths...4830worker1 # [ 100.820776] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347764831worker1 # [ 100.825927] nix-daemon[854]: reaped child process 1270, status = succeeded4832client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4833worker1 # [ 100.852260] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347764834worker1 # [ 100.859021] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=1204835client # [ 100.909529] nix-daemon[1157]: reaped child process 1262, status = succeeded4836client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.71 seconds)4837client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24838client # [ 101.080222] nix-daemon[1157]: accepted connection from pid 1280, user root (trusted)4839client # this derivation will be built:4840client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4841client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4842worker2 # [ 101.234349] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:343584843worker2 # [ 101.240691] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584844worker2 # [ 101.251037] nix-grpc-daemon[866]: time=2026-09-23T05:33:40.866Z level=INFO msg="All 1 paths already cached"4845worker1 # [ 101.339041] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04846worker1 # [ 101.345320] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14847worker2 # [ 101.314942] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584848worker2 # [ 101.330406] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343584849worker2 # [ 101.344671] nix-grpc-daemon[866]: time=2026-09-23T05:33:40.961Z level=INFO msg="All 1 paths already cached"4850client # copying 1 paths to 'lb:50051'...4851client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4852worker2 # [ 101.355902] nix-grpc-daemon[769]: ts=2026-09-23T05:33:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343584853worker2 # [ 101.371164] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4854worker2 # [ 101.532846] nix-daemon[839]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4855worker2 # [ 101.551130] nix-daemon[839]: reaped child process 1113, status = succeeded4856lb # [ 101.548970] niks3-server[975]: 2026/09/23 05:33:41 INFO Received uploads request method=POST path=/api/pending_closures4857worker2 # [ 101.565309] nix-grpc-daemon[866]: time=2026-09-23T05:33:41.181Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4858worker2 # [ 101.566749] nix-grpc-daemon[866]: time=2026-09-23T05:33:41.181Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4859lb # [ 101.583089] niks3-server[975]: 2026/09/23 05:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign4860worker2 # [ 101.596586] nix-grpc-daemon[866]: time=2026-09-23T05:33:41.212Z level=INFO msg="Uploading 1 narinfos"4861lb # [ 101.598214] niks3-server[975]: 2026/09/23 05:33:41 INFO Signed narinfos id=33 count=14862lb # [ 101.605091] niks3-server[975]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4863worker2 # [ 101.623484] nix-grpc-daemon[866]: time=2026-09-23T05:33:41.239Z level=INFO msg="Upload complete. (235ms)"4864lb # [ 101.616968] niks3-server[975]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4865lb # [ 101.624753] niks3-server[975]: 2026/09/23 05:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete4866worker2 # [ 101.632327] nix-grpc-daemon[769]: ts=2026-09-23T05:33:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=0 paths=1 nar_bytes_in=5684867lb # [ 101.630866] niks3-server[975]: 2026/09/23 05:33:41 INFO Completed upload id=334868lb # [ 101.634642] niks3-server[975]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4869worker2 # [ 101.646515] nix-grpc-daemon[769]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343584870worker2 # [ 101.655233] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4871worker2 # [ 101.658236] nix-daemon[839]: reaped child process 1108, status = succeeded4872client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4873worker2 # [ 101.670660] nix-daemon[839]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4874client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4875client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4876worker2 # [ 102.760670] nix-daemon[839]: reaped child process 1117, status = succeeded4877worker2 # [ 102.848961] nix-daemon[839]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)4878worker2 # [ 102.865538] nix-daemon[839]: reaped child process 1136, status = succeeded4879lb # [ 102.864923] niks3-server[975]: 2026/09/23 05:33:42 INFO Received uploads request method=POST path=/api/pending_closures4880worker2 # [ 102.882678] nix-grpc-daemon[866]: time=2026-09-23T05:33:42.497Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4881lb # [ 102.898197] niks3-server[975]: 2026/09/23 05:33:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign4882worker2 # [ 102.916091] nix-grpc-daemon[866]: time=2026-09-23T05:33:42.532Z level=INFO msg="Uploading 1 narinfos"4883lb # [ 102.913495] niks3-server[975]: 2026/09/23 05:33:42 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4884lb # [ 102.924236] niks3-server[975]: 2026/09/23 05:33:42 INFO Signed narinfos id=34 count=14885lb # [ 102.931708] niks3-server[975]: 2026/09/23 05:33:42 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4886worker2 # [ 102.945503] nix-grpc-daemon[866]: time=2026-09-23T05:33:42.561Z level=INFO msg="Upload complete. (264ms)"4887lb # [ 102.939842] niks3-server[975]: 2026/09/23 05:33:42 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete4888lb # [ 102.944198] niks3-server[975]: 2026/09/23 05:33:42 INFO Completed upload id=344889lb # [ 102.947611] niks3-server[975]: 2026/09/23 05:33:42 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4890client # copying 1 paths...4891worker2 # [ 102.957244] nix-grpc-daemon[769]: ts=2026-09-23T05:33:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34358 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=14892worker1 # [ 103.037138] nix-grpc-daemon[769]: ts=2026-09-23T05:33:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347764893worker2 # [ 102.977458] nix-daemon[839]: reaped child process 1115, status = succeeded4894worker1 # [ 103.050064] nix-grpc-daemon[769]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4895client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4896worker1 # [ 103.093573] nix-grpc-daemon[769]: ts=2026-09-23T05:33:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347764897worker1 # [ 103.100594] nix-grpc-daemon[769]: ts=2026-09-23T05:33:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34776 duration_s=0 paths=1 nar_bytes_out=1204898client # [ 103.149501] nix-daemon[1157]: reaped child process 1283, status = succeeded4899client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.24 seconds)4900(finished: subtest: an input only one worker has still reaches the builder, in 8.47 seconds)4901subtest: upload whose reference the worker lacks is completed from the cache4902client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4903client # this derivation will be built:4904client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4905client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4906client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.32 seconds)4907client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4908client # this derivation will be built:4909client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4910client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4911client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)4912client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4913client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)4914worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4915worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)4916worker2 # [ 104.143100] systemd[1]: Stopping Nix worker-protocol over gRPC...4917worker2 # [ 105.057626] nix-grpc-daemon[769]: ts=2026-09-23T05:33:44Z level=info event=signal_exit4918worker2 # [ 106.601274] nix-grpc-daemon[769]: ts=2026-09-23T05:33:46Z level=info event=scheduler_restarting4919worker1 # [ 106.682657] nix-grpc-daemon[769]: ts=2026-09-23T05:33:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:532144920worker1 # [ 106.691597] nix-grpc-daemon[769]: ts=2026-09-23T05:33:46Z level=info event=worker_gone addr=192.168.1.4:500514921worker2 # [ 106.639148] nix-grpc-daemon[769]: ts=2026-09-23T05:33:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4922worker2 # [ 106.665476] nix-daemon[839]: reaped child process 841, status = succeeded4923worker2 # [ 107.207277] nix-grpc-daemon[769]: E0000 00:00:1790141626.823401 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true4924worker2 # [ 107.668552] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4925worker2 # [ 107.676220] systemd[1]: Stopped Nix worker-protocol over gRPC.4926worker2 # [ 107.678846] systemd[1]: nix-grpc-daemon.service: Consumed 2.895s CPU time over 1min 27.887s wall clock time, 60.2M memory peak, 72.3K incoming IP traffic, 124.5K outgoing IP traffic.4927lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux4928lb: must succeed: curl -sf localhost:9901/clusters4929worker2 # [ 107.705525] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4930worker2 # [ 107.707563] systemd[1]: Closed Nix worker-protocol over gRPC.4931worker2 # [ 107.709584] systemd[1]: nix-grpc-daemon.socket: Consumed 520us CPU time over 1min 37.724s wall clock time, 256K memory peak, 89.9K incoming IP traffic, 79.3K outgoing IP traffic.4932lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4933lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.08 seconds)4934client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24935worker1 # [ 108.061603] nix-grpc-daemon[769]: ts=2026-09-23T05:33:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:347764936client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4937client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.24 seconds)4938worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4939worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)4940worker2 # [ 108.118540] systemd[1]: Listening on Nix worker-protocol over gRPC.4941worker1 # [ 108.068760] nix-grpc-daemon[769]: ts=2026-09-23T05:33:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347764942worker1 # [ 108.256838] systemd[1]: Stopping Nix worker-protocol over gRPC...4943worker2 # [ 108.662657] systemd[1]: Starting scheduler-token.service...4944worker2 # [ 108.753518] systemd[1]: scheduler-token.service: Deactivated successfully.4945worker2 # [ 108.760463] systemd[1]: Finished scheduler-token.service.4946worker2 # [ 108.762560] systemd[1]: scheduler-token.service: Consumed 56ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4947worker2 # [ 108.785333] systemd[1]: Starting Nix worker-protocol over gRPC...4948worker2 # [ 108.908666] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4949worker2 # [ 108.910536] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4950worker2 # [ 108.914336] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14951worker2 # [ 108.920086] nix-grpc-daemon[1160]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4952worker2 # [ 108.921496] nix-grpc-daemon[1160]: E0000 00:00:1790141628.535195 1160 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4953worker2 # [ 108.933394] nix-grpc-daemon[1160]: E0000 00:00:1790141628.541412 1160 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4954worker2 # [ 108.968130] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4955worker2 # [ 108.970238] systemd[1]: Started Nix worker-protocol over gRPC.4956worker2 # [ 108.971208] nix-grpc-daemon[1160]: E0000 00:00:1790141628.550902 1201 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4957worker2 # [ 108.973638] nix-grpc-daemon[1160]: E0000 00:00:1790141628.566933 1201 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4958worker2 # [ 108.978546] nix-grpc-daemon[1160]: E0000 00:00:1790141628.567085 1201 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4959worker2 # [ 108.983027] nix-grpc-daemon[1160]: E0000 00:00:1790141628.583752 1201 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4960worker2 # [ 108.985614] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:48Z level=info event=scheduler_connected addr=lb:500514961worker1 # [ 109.055915] nix-grpc-daemon[769]: ts=2026-09-23T05:33:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:532144962worker1 # [ 109.056502] nix-grpc-daemon[769]: ts=2026-09-23T05:33:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04963lb # [ 109.601132] niks3-server[975]: 2026/09/23 05:33:49 INFO lead: released remote=192.168.1.3:538764964worker1 # [ 109.072531] nix-grpc-daemon[769]: ts=2026-09-23T05:33:48Z level=info event=signal_exit4965lb # [ 109.623302] envoy[904]: conn peer=[2001:db8:1::1]:37472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3739 tx=2124 ms=23214966lb # [ 109.628436] envoy[904]: conn peer=[2001:db8:1::1]:37474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=694967lb # [ 109.633108] envoy[904]: conn peer=[2001:db8:1::1]:44196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2099 ms=18534968lb # [ 109.638030] envoy[904]: conn peer=[2001:db8:1::1]:44198 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=684969lb # [ 109.642746] envoy[904]: conn peer=[2001:db8:1::4]:46358 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2543 tx=851 ms=709494970lb # [ 109.647693] envoy[904]: conn peer=[2001:db8:1::1]:44202 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=254971worker1 # [ 110.673183] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=scheduler_restarting4972worker2 # [ 110.605105] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_restarting addr=lb:500514973lb # [ 110.594482] envoy[904]: [2026-09-23 05:33:50.197][904][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}4974worker1 # [ 110.682868] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:532144975worker1 # [ 110.694424] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=worker_gone addr=192.168.1.4:500514976worker1 # [ 110.698881] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=scheduler_restarting addr=lb:500514977worker1 # [ 110.705253] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:532144978worker1 # [ 110.710370] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=worker_gone addr=192.168.1.3:500514979worker1 # [ 110.735064] nix-daemon[854]: reaped child process 856, status = succeeded4980worker2 # [ 110.805559] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_connected addr=lb:500514981worker2 # [ 110.810942] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4982worker1 # [ 110.916585] nix-grpc-daemon[769]: ts=2026-09-23T05:33:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4983worker2 # [ 111.011856] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_connected addr=lb:500514984worker2 # [ 111.017214] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4985worker1 # [ 111.122062] nix-grpc-daemon[769]: E0000 00:00:1790141630.725870 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true4986worker1 # [ 111.243449] nix-grpc-daemon[769]: E0000 00:00:1790141630.847365 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4987worker2 # [ 111.218579] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_connected addr=lb:500514988worker2 # [ 111.223742] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4989worker2 # [ 111.424941] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:500514990worker2 # [ 111.429988] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4991worker1 # [ 111.686942] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4992worker1 # [ 111.694720] systemd[1]: Stopped Nix worker-protocol over gRPC.4993worker2 # [ 111.632933] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:500514994worker1 # [ 111.703156] systemd[1]: nix-grpc-daemon.service: Consumed 3.789s CPU time over 1min 31.982s wall clock time, 73.1M memory peak, 98.8K incoming IP traffic, 201.3K outgoing IP traffic.4995worker2 # [ 111.637040] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4996worker1 # [ 111.719547] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4997worker1 # [ 111.725313] systemd[1]: Closed Nix worker-protocol over gRPC.4998worker1 # [ 111.727056] systemd[1]: nix-grpc-daemon.socket: Consumed 518us CPU time over 1min 41.759s wall clock time, 256K memory peak, 359K incoming IP traffic, 110.6K outgoing IP traffic.4999lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5000lb: must succeed: curl -sf localhost:9901/clusters5001lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5002lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.07 seconds)5003client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25004worker2 # [ 111.833817] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:500515005worker2 # [ 111.839007] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5006worker2 # [ 111.956076] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:451405007worker2 # [ 111.971197] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5008worker2 # [ 111.995265] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 1/5)5009worker2 # [ 112.022583] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 2/5)5010worker2 # [ 112.040159] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:500515011worker2 # [ 112.045154] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5012worker2 # [ 112.163594] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 159 ms (attempt 3/5)5013worker2 # [ 112.246284] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_connected addr=lb:500515014worker2 # [ 112.251401] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5015worker2 # [ 112.324505] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 596 ms (attempt 4/5)5016worker2 # [ 112.452659] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:500515017worker2 # [ 112.457692] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5018worker2 # [ 112.660501] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:500515019worker2 # [ 112.664607] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5020worker2 # [ 112.861387] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:500515021worker2 # [ 112.867093] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5022worker2 # [ 112.922990] nix-grpc-daemon[1160]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5023worker2 # [ 112.930957] nix-grpc-daemon[1160]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5024client # copying 2 paths...5025client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5026worker2 # [ 112.953738] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:451405027client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5028worker2 # [ 112.972101] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5029worker2 # [ 113.001963] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=niks3_push_started pid=12215030worker2 # [ 113.068699] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:500515031worker2 # [ 113.075791] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5032worker2 # [ 113.166943] nix-grpc-daemon[1221]: time=2026-09-23T05:33:52.782Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5033worker2 # [ 113.273061] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_connected addr=lb:500515034worker2 # [ 113.277248] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5035worker2 # [ 113.282299] nix-daemon[839]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5036worker2 # [ 113.300708] nix-daemon[839]: reaped child process 1236, status = succeeded5037lb # [ 113.301810] niks3-server[975]: 2026/09/23 05:33:52 INFO Received uploads request method=POST path=/api/pending_closures5038lb # [ 113.312139] niks3-server[975]: 2026/09/23 05:33:52 INFO Received uploads request method=POST path=/api/pending_closures5039worker2 # [ 113.330047] nix-grpc-daemon[1221]: time=2026-09-23T05:33:52.944Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5040worker2 # [ 113.331519] nix-grpc-daemon[1221]: time=2026-09-23T05:33:52.944Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5041worker2 # [ 113.335034] nix-grpc-daemon[1221]: time=2026-09-23T05:33:52.944Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5042lb # [ 113.381500] niks3-server[975]: 2026/09/23 05:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5043lb # [ 113.396359] niks3-server[975]: 2026/09/23 05:33:52 INFO Signed narinfos id=35 count=15044worker2 # [ 113.404103] nix-grpc-daemon[1221]: time=2026-09-23T05:33:53.019Z level=INFO msg="Uploading 3 narinfos"5045lb # [ 113.403815] niks3-server[975]: 2026/09/23 05:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5046lb # [ 113.413925] niks3-server[975]: 2026/09/23 05:33:52 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5047lb # [ 113.422912] niks3-server[975]: 2026/09/23 05:33:52 INFO Signed narinfos id=36 count=25048lb # [ 113.426942] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5049lb # [ 113.436117] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5050lb # [ 113.442254] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5051lb # [ 113.453685] niks3-server[975]: 2026/09/23 05:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5052lb # [ 113.464261] niks3-server[975]: 2026/09/23 05:33:53 INFO Completed upload id=355053lb # [ 113.468623] niks3-server[975]: 2026/09/23 05:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5054worker2 # [ 113.476596] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:500515055lb # [ 113.472963] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5056lb # [ 113.475269] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5057worker2 # [ 113.482114] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5058lb # [ 113.479881] niks3-server[975]: 2026/09/23 05:33:53 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5059lb # [ 113.482740] niks3-server[975]: 2026/09/23 05:33:53 INFO Completed upload id=365060worker2 # [ 113.488580] nix-grpc-daemon[1221]: time=2026-09-23T05:33:53.103Z level=INFO msg="Upload complete. (317ms)"5061worker2 # [ 113.500461] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=2 nar_bytes_in=3125062worker2 # [ 113.516906] nix-daemon[839]: reaped child process 1219, status = succeeded5063client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.80 seconds)5064worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5065worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5066worker1 # [ 113.709135] systemd[1]: Listening on Nix worker-protocol over gRPC.5067worker1 # [ 113.717806] systemd[1]: Starting Nix worker-protocol over gRPC...5068worker2 # [ 113.684577] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:500515069worker2 # [ 113.689392] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5070worker1 # [ 113.856707] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5071worker1 # [ 113.858607] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5072worker1 # [ 113.862559] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15073worker1 # [ 113.868060] nix-grpc-daemon[1317]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5074worker1 # [ 113.869522] nix-grpc-daemon[1317]: E0000 00:00:1790141633.471487 1317 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5075lb # [ 113.803559] niks3-server[975]: 2026/09/23 05:33:53 INFO lead: acquired remote=192.168.1.3:370245076lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5077lb: must succeed: curl -sf localhost:9901/clusters5078worker1 # [ 113.914444] nix-grpc-daemon[1317]: E0000 00:00:1790141633.477632 1317 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5079worker1 # [ 113.917864] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5080worker1 # [ 113.920139] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_take_over5081worker1 # [ 113.921420] nix-grpc-daemon[1317]: E0000 00:00:1790141633.485599 1357 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5082worker1 # [ 113.923763] nix-grpc-daemon[1317]: E0000 00:00:1790141633.502714 1357 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5083worker1 # [ 113.929249] nix-grpc-daemon[1317]: E0000 00:00:1790141633.502881 1357 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5084worker1 # [ 113.931615] nix-grpc-daemon[1317]: E0000 00:00:1790141633.518419 1357 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5085worker1 # [ 113.936318] systemd[1]: Started Nix worker-protocol over gRPC.5086worker1 # [ 113.937166] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:500515087worker1 # [ 113.938457] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5088worker2 # [ 113.890584] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:500515089worker2 # [ 113.895887] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5090lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5091worker1 # [ 114.137634] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:500515092worker1 # [ 114.142841] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5093worker2 # [ 114.097000] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:500515094worker2 # [ 114.102126] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5095worker1 # [ 114.343610] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:500515096worker1 # [ 114.349075] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5097worker2 # [ 114.303256] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_connected addr=lb:500515098worker2 # [ 114.308533] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5099worker1 # [ 114.549811] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=scheduler_connected addr=lb:500515100worker1 # [ 114.554960] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5101worker2 # [ 114.509697] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:54Z level=info event=scheduler_connected addr=lb:500515102worker2 # [ 114.514701] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5103worker1 # [ 114.755595] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=scheduler_connected addr=lb:500515104worker1 # [ 114.764191] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:573525105worker1 # [ 114.769427] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05106worker2 # [ 114.717186] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:54Z level=info event=scheduler_connected addr=lb:500515107worker1 # [ 114.795240] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=oidc_jwks provider=mock keys=15108worker1 # [ 114.799758] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:573525109worker1 # [ 114.805314] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05110lb: must succeed: curl -sf localhost:9901/clusters5111lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5112lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.14 seconds)5113(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.80 seconds)5114subtest: a niks3 restart does not cost the leader its role5115lb: must succeed: curl -sf localhost:9901/clusters5116lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5117lb: must succeed: curl -sf localhost:9901/clusters5118lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5119lb # [ 115.059306] systemd[1]: Stopping niks3 server...5120worker2 # [ 115.067339] nix-grpc-daemon[1160]: ts=2026-09-23T05:33:54Z level=info event=scheduler_lock_lost status=2005121worker1 # [ 115.137603] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:54Z level=info event=scheduler_lock_lost status=2005122lb # [ 115.066861] niks3-server[975]: 2026/09/23 05:33:54 INFO Shutdown signal received, draining in-flight requests timeout=10s5123lb # [ 115.075374] niks3-server[975]: 2026/09/23 05:33:54 INFO lead: released remote=192.168.1.3:370245124lb # [ 115.081728] systemd[1]: niks3.service: Deactivated successfully.5125lb # [ 115.083881] systemd[1]: Stopped niks3 server.5126lb # [ 115.085757] systemd[1]: niks3.service: Consumed 643ms CPU time over 1min 30.734s wall clock time, 27.4M memory peak, 90.1K incoming IP traffic, 132.9K outgoing IP traffic.5127lb # [ 115.094024] systemd[1]: Starting niks3 server...5128lb # [ 115.155137] niks3-server[1137]: 2026/09/23 05:33:54 goose: no migrations to run. current version: 202609200000005129lb # [ 115.160918] niks3-server[1137]: 2026/09/23 05:33:54 OK 1_commit_pending_closure.sql (7.24ms)5130lb # [ 115.164375] niks3-server[1137]: 2026/09/23 05:33:54 OK 2_object_stats_trigger.sql (3.09ms)5131lb # [ 115.166469] niks3-server[1137]: 2026/09/23 05:33:54 goose: up to current file version: 25132lb # [ 115.167971] niks3-server[1137]: 2026/09/23 05:33:54 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5133lb # [ 115.177702] niks3-server[1137]: 2026/09/23 05:33:54 INFO Read proxy enabled — serving cache objects from S35134lb # [ 115.179033] niks3-server[1137]: 2026/09/23 05:33:54 INFO Using socket-activated listener address=0.0.0.0:57515135lb # [ 115.183560] niks3-server[1137]: 2026/09/23 05:33:54 INFO systemd watchdog enabled interval=15s5136lb # [ 115.184906] niks3-server[1137]: 2026/09/23 05:33:54 INFO Starting HTTP server address=0.0.0.0:57515137lb # [ 115.187095] systemd[1]: Started niks3 server.5138lb: waiting for unit niks3.service5139lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5140lb: must succeed: curl -sf localhost:9901/clusters5141lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5142worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5143worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5144lb: must succeed: curl -sf localhost:9901/clusters5145lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5146lb: must succeed: curl -sf localhost:9901/clusters5147lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5148worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5149worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5150client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25151worker1 # [ 115.736133] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:573665152worker1 # [ 115.752458] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5153client # don't know how to build these paths:5154client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5155worker1 # [ 115.774742] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:573665156worker1 # [ 115.791100] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)5157worker1 # [ 115.826377] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 2/5)5158worker1 # [ 115.925257] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 3/5)5159worker1 # [ 116.024069] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 576 ms (attempt 4/5)5160lb # [ 116.066314] niks3-server[1137]: 2026/09/23 05:33:55 INFO lead: acquired remote=192.168.1.3:370365161worker1 # [ 116.605774] nix-grpc-daemon[1317]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5162worker1 # [ 116.612709] nix-grpc-daemon[1317]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5163worker1 # [ 116.657552] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05164worker1 # [ 116.664851] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=15165worker1 # [ 116.705511] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:573665166worker1 # [ 116.723749] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:573665167client # copying 1 paths to 'lb:50051'...5168client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5169worker1 # [ 116.738151] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:573665170worker1 # [ 116.753389] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5171worker1 # [ 116.781933] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=info event=niks3_push_started pid=13905172worker1 # [ 116.931184] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.534Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5173worker1 # [ 117.037219] nix-daemon[854]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)5174worker1 # [ 117.054831] nix-daemon[854]: reaped child process 1406, status = succeeded5175lb # [ 116.986907] niks3-server[1137]: 2026/09/23 05:33:56 INFO Received uploads request method=POST path=/api/pending_closures5176worker1 # [ 117.079419] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.682Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5177worker1 # [ 117.083351] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.682Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5178lb # [ 117.030857] niks3-server[1137]: 2026/09/23 05:33:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5179lb # [ 117.066364] niks3-server[1137]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5180lb # [ 117.082507] niks3-server[1137]: 2026/09/23 05:33:56 INFO Signed narinfos id=37 count=15181worker1 # [ 117.163446] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.765Z level=INFO msg="Uploading 1 narinfos"5182lb # [ 117.095467] niks3-server[1137]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5183lb # [ 117.108069] niks3-server[1137]: 2026/09/23 05:33:56 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5184lb # [ 117.114769] niks3-server[1137]: 2026/09/23 05:33:56 INFO Completed upload id=375185worker1 # [ 117.191587] nix-grpc-daemon[1390]: time=2026-09-23T05:33:56.795Z level=INFO msg="Upload complete. (256ms)"5186lb # [ 117.120299] niks3-server[1137]: 2026/09/23 05:33:56 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5187worker1 # [ 117.203575] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=0 paths=1 nar_bytes_in=5445188worker1 # [ 117.216824] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:573665189worker1 # [ 117.231561] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5190worker1 # [ 117.235755] nix-daemon[854]: reaped child process 1388, status = succeeded5191client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5192worker1 # [ 117.246341] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5193client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5194client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5195client # farm-a> LOG-farm-a-after-niks3-restart5196worker1 # [ 118.101928] nix-daemon[854]: reaped child process 1410, status = succeeded5197worker1 # [ 118.228618] nix-daemon[854]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5198worker1 # [ 118.246452] nix-daemon[854]: reaped child process 1429, status = succeeded5199lb # [ 118.177766] niks3-server[1137]: 2026/09/23 05:33:57 INFO Received uploads request method=POST path=/api/pending_closures5200worker1 # [ 118.267747] nix-grpc-daemon[1390]: time=2026-09-23T05:33:57.871Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5201worker1 # [ 118.272979] nix-grpc-daemon[1390]: time=2026-09-23T05:33:57.871Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5202lb # [ 118.226693] niks3-server[1137]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5203lb # [ 118.240105] niks3-server[1137]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5204worker1 # [ 118.329588] nix-grpc-daemon[1390]: time=2026-09-23T05:33:57.932Z level=INFO msg="Uploading 1 narinfos"5205lb # [ 118.246921] niks3-server[1137]: 2026/09/23 05:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5206lb # [ 118.257218] niks3-server[1137]: 2026/09/23 05:33:57 INFO Signed narinfos id=38 count=15207lb # [ 118.267257] niks3-server[1137]: 2026/09/23 05:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5208worker1 # [ 118.350860] nix-grpc-daemon[1390]: time=2026-09-23T05:33:57.954Z level=INFO msg="Upload complete. (292ms)"5209lb # [ 118.277734] niks3-server[1137]: 2026/09/23 05:33:57 INFO Completed upload id=385210lb # [ 118.279695] niks3-server[1137]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5211lb # [ 118.285456] niks3-server[1137]: 2026/09/23 05:33:57 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5212worker1 # [ 118.374122] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15213worker1 # [ 118.390614] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:57Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05214worker1 # [ 118.394830] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:57Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=15215worker1 # [ 118.399112] nix-daemon[854]: reaped child process 1408, status = succeeded5216worker1 # [ 118.418230] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:573665217worker1 # [ 118.433806] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:573665218worker1 # [ 118.449408] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.053Z level=INFO msg="All 1 paths already cached"5219client # copying 1 paths to 'lb:50051'...5220client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5221worker1 # [ 118.459666] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:573665222worker1 # [ 118.473728] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5223worker1 # [ 118.626666] nix-daemon[854]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5224worker1 # [ 118.643537] nix-daemon[854]: reaped child process 1438, status = succeeded5225lb # [ 118.572452] niks3-server[1137]: 2026/09/23 05:33:58 INFO Received uploads request method=POST path=/api/pending_closures5226worker1 # [ 118.658346] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.262Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5227worker1 # [ 118.659817] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.262Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5228lb # [ 118.600706] niks3-server[1137]: 2026/09/23 05:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5229worker1 # [ 118.687161] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.290Z level=INFO msg="Uploading 1 narinfos"5230lb # [ 118.616541] niks3-server[1137]: 2026/09/23 05:33:58 INFO Signed narinfos id=39 count=15231lb # [ 118.619911] niks3-server[1137]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5232lb # [ 118.633071] niks3-server[1137]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5233lb # [ 118.647942] niks3-server[1137]: 2026/09/23 05:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5234lb # [ 118.654013] niks3-server[1137]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5235worker1 # [ 118.730921] nix-grpc-daemon[1390]: time=2026-09-23T05:33:58.334Z level=INFO msg="Upload complete. (241ms)"5236lb # [ 118.658101] niks3-server[1137]: 2026/09/23 05:33:58 INFO Completed upload id=395237worker1 # [ 118.741007] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=0 paths=1 nar_bytes_in=6565238worker1 # [ 118.756514] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:573665239worker1 # [ 118.768216] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5240worker1 # [ 118.771355] nix-daemon[854]: reaped child process 1433, status = succeeded5241client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5242worker1 # [ 118.784254] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5243client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5244client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5245worker1 # [ 119.572085] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:573685246lb # [ 119.624098] envoy[904]: conn peer=192.168.1.3:57412 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2491 tx=1124 ms=749805247client # farm-b> LOG-farm-b-after-niks3-restart5248worker1 # [ 119.829011] nix-daemon[854]: reaped child process 1442, status = succeeded5249worker1 # [ 119.936749] nix-daemon[854]: accepted connection from pid 1458, user nix-grpc-daemon (trusted)5250worker1 # [ 119.954176] nix-daemon[854]: reaped child process 1461, status = succeeded5251lb # [ 119.629170] envoy[904]: conn peer=[2001:db8:1::1]:41590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=282 ms=15595252lb # [ 119.882411] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received uploads request method=POST path=/api/pending_closures5253worker1 # [ 119.979697] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.582Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5254worker1 # [ 119.981698] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.582Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5255lb # [ 119.930973] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5256worker1 # [ 120.015503] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.619Z level=INFO msg="Uploading 1 narinfos"5257lb # [ 119.946681] niks3-server[1137]: 2026/09/23 05:33:59 INFO Signed narinfos id=40 count=15258lb # [ 119.957552] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5259lb # [ 119.967074] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5260lb # [ 119.976238] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5261worker1 # [ 120.052980] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.656Z level=INFO msg="Upload complete. (256ms)"5262lb # [ 119.979298] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5263lb # [ 119.985234] niks3-server[1137]: 2026/09/23 05:33:59 INFO Completed upload id=405264lb # [ 119.988211] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5265worker1 # [ 120.067334] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15266worker1 # [ 120.089817] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05267worker1 # [ 120.094648] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=15268worker1 # [ 120.100200] nix-daemon[854]: reaped child process 1440, status = succeeded5269worker1 # [ 120.129253] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:573665270worker1 # [ 120.144955] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:573665271worker1 # [ 120.160031] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.763Z level=INFO msg="All 2 paths already cached"5272client # copying 1 paths to 'lb:50051'...5273client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5274worker1 # [ 120.171008] nix-grpc-daemon[1317]: ts=2026-09-23T05:33:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:573665275worker1 # [ 120.184440] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5276worker1 # [ 120.338973] nix-daemon[854]: accepted connection from pid 1467, user nix-grpc-daemon (trusted)5277worker1 # [ 120.358819] nix-daemon[854]: reaped child process 1470, status = succeeded5278lb # [ 120.285399] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received uploads request method=POST path=/api/pending_closures5279worker1 # [ 120.371403] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.975Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5280worker1 # [ 120.373516] nix-grpc-daemon[1390]: time=2026-09-23T05:33:59.975Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5281lb # [ 120.317321] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5282worker1 # [ 120.403683] nix-grpc-daemon[1390]: time=2026-09-23T05:34:00.007Z level=INFO msg="Uploading 1 narinfos"5283lb # [ 120.327982] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5284lb # [ 120.338382] niks3-server[1137]: 2026/09/23 05:33:59 INFO Signed narinfos id=41 count=15285lb # [ 120.343866] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5286worker1 # [ 120.433714] nix-grpc-daemon[1390]: time=2026-09-23T05:34:00.037Z level=INFO msg="Upload complete. (233ms)"5287lb # [ 120.357779] niks3-server[1137]: 2026/09/23 05:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5288lb # [ 120.363337] niks3-server[1137]: 2026/09/23 05:33:59 INFO Completed upload id=415289lb # [ 120.366875] niks3-server[1137]: 2026/09/23 05:33:59 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5290worker1 # [ 120.442776] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=0 paths=1 nar_bytes_in=7845291worker1 # [ 120.459648] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:573665292worker1 # [ 120.466258] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5293worker1 # [ 120.472322] nix-daemon[854]: reaped child process 1465, status = succeeded5294client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5295worker1 # [ 120.485230] nix-daemon[854]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5296client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5297client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5298client # farm-top> LOG-farm-top-after-niks3-restart5299worker1 # [ 121.915971] nix-daemon[854]: reaped child process 1474, status = succeeded5300worker1 # [ 122.010712] nix-daemon[854]: accepted connection from pid 1490, user nix-grpc-daemon (trusted)5301worker1 # [ 122.028310] nix-daemon[854]: reaped child process 1493, status = succeeded5302lb # [ 121.958719] niks3-server[1137]: 2026/09/23 05:34:01 INFO Received uploads request method=POST path=/api/pending_closures5303worker1 # [ 122.047254] nix-grpc-daemon[1390]: time=2026-09-23T05:34:01.649Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5304worker1 # [ 122.048902] nix-grpc-daemon[1390]: time=2026-09-23T05:34:01.649Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5305lb # [ 121.999631] niks3-server[1137]: 2026/09/23 05:34:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5306worker1 # [ 122.085771] nix-grpc-daemon[1390]: time=2026-09-23T05:34:01.689Z level=INFO msg="Uploading 1 narinfos"5307lb # [ 122.013952] niks3-server[1137]: 2026/09/23 05:34:01 INFO Signed narinfos id=42 count=15308lb # [ 122.018525] niks3-server[1137]: 2026/09/23 05:34:01 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5309lb # [ 122.030421] niks3-server[1137]: 2026/09/23 05:34:01 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5310lb # [ 122.040583] niks3-server[1137]: 2026/09/23 05:34:01 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5311lb # [ 122.043382] niks3-server[1137]: 2026/09/23 05:34:01 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5312lb # [ 122.049994] niks3-server[1137]: 2026/09/23 05:34:01 INFO Completed upload id=425313worker1 # [ 122.126569] nix-grpc-daemon[1390]: time=2026-09-23T05:34:01.730Z level=INFO msg="Upload complete. (231ms)"5314lb # [ 122.052623] niks3-server[1137]: 2026/09/23 05:34:01 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5315worker1 # [ 122.141013] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57366 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15316worker1 # [ 122.155305] nix-daemon[854]: reaped child process 1472, status = succeeded5317client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.66 seconds)5318client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5319client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5320(finished: subtest: a niks3 restart does not cost the leader its role, in 7.31 seconds)5321subtest: scheduler down: the other node takes the lock and keeps it5322lb: must succeed: curl -sf localhost:9901/clusters5323lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5324lb: must succeed: curl -sf localhost:9901/clusters5325lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5326lb: must succeed: curl -sf localhost:9901/clusters5327lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5328lb: must succeed: curl -sf localhost:9901/clusters5329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5330worker1 # [ 122.601142] systemd[1]: Stopping Nix worker-protocol over gRPC...5331worker1 # [ 122.883133] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:02Z level=info event=signal_exit5332lb # [ 123.075405] niks3-server[1137]: 2026/09/23 05:34:02 INFO lead: released remote=192.168.1.3:370365333worker1 # [ 124.152932] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=scheduler_restarting5334worker2 # [ 124.085308] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:03Z level=info event=scheduler_restarting addr=lb:500515335lb # [ 124.075585] envoy[904]: [2026-09-23 05:34:03.677][904][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5336worker1 # [ 124.158472] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=scheduler_restarting addr=lb:500515337worker1 # [ 124.168868] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573525338worker1 # [ 124.174282] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=worker_gone addr=192.168.1.3:500515339worker1 # [ 124.178596] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573525340worker1 # [ 124.183574] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=worker_gone addr=192.168.1.4:500515341worker1 # [ 124.207950] nix-daemon[854]: reaped child process 1378, status = succeeded5342worker2 # [ 124.290965] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:03Z level=info event=scheduler_connected addr=lb:500515343worker2 # [ 124.296280] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5344worker1 # [ 124.378981] nix-grpc-daemon[1317]: ts=2026-09-23T05:34:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5345worker2 # [ 124.497191] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:500515346worker2 # [ 124.502346] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker1 # [ 124.724081] nix-grpc-daemon[1317]: E0000 00:00:1790141644.327706 1363 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 23, started true, shutdown true5348worker1 # [ 124.730701] nix-grpc-daemon[1317]: E0000 00:00:1790141644.328016 1363 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5349worker2 # [ 124.703379] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:500515350worker2 # [ 124.708597] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5351worker2 # [ 124.909763] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:500515352worker2 # [ 124.914943] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker2 # [ 125.117615] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:500515354worker2 # [ 125.121684] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker2 # [ 125.318258] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_connected addr=lb:500515356worker2 # [ 125.323319] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357worker1 # [ 125.422324] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5358worker1 # [ 125.422591] systemd[1]: Stopped Nix worker-protocol over gRPC.5359worker1 # [ 125.422745] systemd[1]: nix-grpc-daemon.service: Consumed 1.203s CPU time over 11.701s wall clock time, 38.7M memory peak, 36.8K incoming IP traffic, 59.1K outgoing IP traffic.5360worker1 # [ 125.444349] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5361worker1 # [ 125.450926] systemd[1]: Closed Nix worker-protocol over gRPC.5362worker1 # [ 125.455709] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 11.753s wall clock time, 256K memory peak, 152K incoming IP traffic, 27.2K outgoing IP traffic.5363worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5364worker2 # [ 125.524511] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:500515365worker2 # [ 125.529583] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366worker2 # [ 125.730551] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:500515367worker2 # [ 125.735810] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5368worker2 # [ 125.937267] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:500515369worker2 # [ 125.942401] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5370lb # [ 126.068611] niks3-server[1137]: 2026/09/23 05:34:05 INFO lead: acquired remote=192.168.1.4:581725371worker2 # [ 126.079814] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_take_over5372worker2 # [ 126.145050] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:500515373worker2 # [ 126.149156] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374worker2 # [ 126.345914] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_connected addr=lb:500515375worker2 # [ 126.350908] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.14 seconds)5377worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5378worker2 # [ 126.552364] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_connected addr=lb:500515379worker2 # [ 126.560639] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5380worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5381worker2 # [ 126.761788] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_connected addr=lb:500515382worker2 # [ 126.767003] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5383worker2 # [ 126.967950] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=scheduler_connected addr=lb:500515384worker2 # [ 126.996143] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=oidc_jwks provider=mock keys=15385worker2 # [ 127.002063] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:499185386worker2 # [ 127.007532] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05387worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5388worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5389client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25390worker2 # [ 127.879841] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:451405391client # don't know how to build these paths:5392client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5393worker2 # [ 127.890336] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:451405394worker2 # [ 127.918509] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05395worker2 # [ 127.925975] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15396worker2 # [ 127.963039] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:451405397worker2 # [ 127.980302] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:451405398client # copying 1 paths to 'lb:50051'...5399client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5400worker2 # [ 127.995812] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:451405401worker2 # [ 128.010823] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5402worker2 # [ 128.169227] nix-daemon[839]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5403worker2 # [ 128.189746] nix-daemon[839]: reaped child process 1268, status = succeeded5404lb # [ 128.187705] niks3-server[1137]: 2026/09/23 05:34:07 INFO Received uploads request method=POST path=/api/pending_closures5405worker2 # [ 128.204398] nix-grpc-daemon[1221]: time=2026-09-23T05:34:07.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5406worker2 # [ 128.205854] nix-grpc-daemon[1221]: time=2026-09-23T05:34:07.820Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5407lb # [ 128.218905] niks3-server[1137]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5408worker2 # [ 128.242778] nix-grpc-daemon[1221]: time=2026-09-23T05:34:07.858Z level=INFO msg="Uploading 1 narinfos"5409lb # [ 128.228915] niks3-server[1137]: 2026/09/23 05:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5410lb # [ 128.241485] niks3-server[1137]: 2026/09/23 05:34:07 INFO Signed narinfos id=43 count=15411lb # [ 128.248682] niks3-server[1137]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5412lb # [ 128.263760] niks3-server[1137]: 2026/09/23 05:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5413lb # [ 128.271943] niks3-server[1137]: 2026/09/23 05:34:07 INFO Completed upload id=435414worker2 # [ 128.278763] nix-grpc-daemon[1221]: time=2026-09-23T05:34:07.895Z level=INFO msg="Upload complete. (251ms)"5415lb # [ 128.275063] niks3-server[1137]: 2026/09/23 05:34:07 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5416worker2 # [ 128.288307] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=1 nar_bytes_in=5205417worker2 # [ 128.306462] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:451405418worker2 # [ 128.313879] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5419worker2 # [ 128.319399] nix-daemon[839]: reaped child process 1263, status = succeeded5420client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5421worker2 # [ 128.333281] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5422client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5423worker2 # [ 128.766629] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:08Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:499245424client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5425client # farm-a> LOG-farm-a-on-standby5426worker2 # [ 129.433067] nix-daemon[839]: reaped child process 1272, status = succeeded5427worker2 # [ 129.512748] nix-daemon[839]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)5428worker2 # [ 129.535855] nix-daemon[839]: reaped child process 1291, status = succeeded5429lb # [ 129.527938] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received uploads request method=POST path=/api/pending_closures5430worker2 # [ 129.548352] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.162Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5431worker2 # [ 129.549940] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.162Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5432lb # [ 129.573998] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5433worker2 # [ 129.593837] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.209Z level=INFO msg="Uploading 1 narinfos"5434lb # [ 129.590620] niks3-server[1137]: 2026/09/23 05:34:09 INFO Signed narinfos id=44 count=15435lb # [ 129.603355] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5436lb # [ 129.613144] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5437lb # [ 129.630524] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5438lb # [ 129.636880] envoy[904]: conn peer=[2001:db8:1::1]:41600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9753 tx=4458 ms=64185439lb # [ 129.639877] envoy[904]: conn peer=[2001:db8:1::3]:45224 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1605 tx=597 ms=104385440lb # [ 129.644457] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5441worker2 # [ 129.650453] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.266Z level=INFO msg="Upload complete. (303ms)"5442lb # [ 129.648798] niks3-server[1137]: 2026/09/23 05:34:09 INFO Completed upload id=445443lb # [ 129.653096] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5444worker2 # [ 129.672948] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15445worker2 # [ 129.691940] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05446worker2 # [ 129.695909] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15447worker2 # [ 129.700756] nix-daemon[839]: reaped child process 1270, status = succeeded5448worker2 # [ 129.715658] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:451405449worker2 # [ 129.731744] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:451405450worker2 # [ 129.746885] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.363Z level=INFO msg="All 1 paths already cached"5451client # copying 1 paths to 'lb:50051'...5452client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5453worker2 # [ 129.758208] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:451405454worker2 # [ 129.772241] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5455worker2 # [ 129.923894] nix-daemon[839]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5456worker2 # [ 129.946555] nix-daemon[839]: reaped child process 1300, status = succeeded5457lb # [ 129.937980] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received uploads request method=POST path=/api/pending_closures5458worker2 # [ 129.955577] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.571Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5459worker2 # [ 129.957379] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.571Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5460lb # [ 129.984226] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5461worker2 # [ 130.002724] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.618Z level=INFO msg="Uploading 1 narinfos"5462lb # [ 129.999829] niks3-server[1137]: 2026/09/23 05:34:09 INFO Signed narinfos id=45 count=15463lb # [ 130.007798] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5464lb # [ 130.014018] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5465lb # [ 130.024088] niks3-server[1137]: 2026/09/23 05:34:09 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5466lb # [ 130.029703] niks3-server[1137]: 2026/09/23 05:34:09 INFO Completed upload id=455467worker2 # [ 130.035539] nix-grpc-daemon[1221]: time=2026-09-23T05:34:09.651Z level=INFO msg="Upload complete. (247ms)"5468lb # [ 130.033002] niks3-server[1137]: 2026/09/23 05:34:09 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5469worker2 # [ 130.045693] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=1 nar_bytes_in=6405470worker2 # [ 130.065809] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:451405471worker2 # [ 130.078274] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5472worker2 # [ 130.083227] nix-daemon[839]: reaped child process 1295, status = succeeded5473client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5474worker2 # [ 130.095973] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5475client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5476client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5477client # farm-b> LOG-farm-b-on-standby5478worker2 # [ 131.380426] nix-daemon[839]: reaped child process 1304, status = succeeded5479worker2 # [ 131.479093] nix-daemon[839]: accepted connection from pid 1320, user nix-grpc-daemon (trusted)5480worker2 # [ 131.498843] nix-daemon[839]: reaped child process 1323, status = succeeded5481lb # [ 131.497435] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received uploads request method=POST path=/api/pending_closures5482worker2 # [ 131.515162] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.130Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5483worker2 # [ 131.517083] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.130Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5484lb # [ 131.541376] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5485worker2 # [ 131.555701] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.171Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 131.554113] niks3-server[1137]: 2026/09/23 05:34:11 INFO Signed narinfos id=46 count=15487lb # [ 131.559886] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5488lb # [ 131.567646] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5489lb # [ 131.576926] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5490lb # [ 131.584286] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5491lb # [ 131.591725] niks3-server[1137]: 2026/09/23 05:34:11 INFO Completed upload id=465492worker2 # [ 131.597524] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.213Z level=INFO msg="Upload complete. (286ms)"5493lb # [ 131.593858] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5494worker2 # [ 131.616865] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15495worker2 # [ 131.632390] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05496worker2 # [ 131.637883] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15497worker2 # [ 131.642238] nix-daemon[839]: reaped child process 1302, status = succeeded5498worker2 # [ 131.659198] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:451405499worker2 # [ 131.674695] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:451405500worker2 # [ 131.689250] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.305Z level=INFO msg="All 2 paths already cached"5501client # copying 1 paths to 'lb:50051'...5502client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5503worker2 # [ 131.700100] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:451405504worker2 # [ 131.713540] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5505worker2 # [ 131.866012] nix-daemon[839]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5506worker2 # [ 131.882975] nix-daemon[839]: reaped child process 1332, status = succeeded5507lb # [ 131.884517] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received uploads request method=POST path=/api/pending_closures5508worker2 # [ 131.901943] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.518Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5509worker2 # [ 131.903515] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.518Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5510lb # [ 131.915789] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5511worker2 # [ 131.928532] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.544Z level=INFO msg="Uploading 1 narinfos"5512lb # [ 131.925901] niks3-server[1137]: 2026/09/23 05:34:11 INFO Signed narinfos id=47 count=15513lb # [ 131.935293] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5514worker2 # [ 131.955421] nix-grpc-daemon[1221]: time=2026-09-23T05:34:11.571Z level=INFO msg="Upload complete. (225ms)"5515lb # [ 131.948786] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5516lb # [ 131.954385] niks3-server[1137]: 2026/09/23 05:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5517lb # [ 131.959886] niks3-server[1137]: 2026/09/23 05:34:11 INFO Completed upload id=475518worker2 # [ 131.964269] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=1 nar_bytes_in=7685519lb # [ 131.962049] niks3-server[1137]: 2026/09/23 05:34:11 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5520worker2 # [ 131.977016] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:451405521worker2 # [ 131.985604] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5522worker2 # [ 131.990827] nix-daemon[839]: reaped child process 1327, status = succeeded5523client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5524worker2 # [ 132.000912] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5525client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5526client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5527client # farm-top> LOG-farm-top-on-standby5528worker2 # [ 133.040274] nix-daemon[839]: reaped child process 1336, status = succeeded5529worker2 # [ 133.134259] nix-daemon[839]: accepted connection from pid 1352, user nix-grpc-daemon (trusted)5530worker2 # [ 133.153278] nix-daemon[839]: reaped child process 1355, status = succeeded5531lb # [ 133.152981] niks3-server[1137]: 2026/09/23 05:34:12 INFO Received uploads request method=POST path=/api/pending_closures5532worker2 # [ 133.171671] nix-grpc-daemon[1221]: time=2026-09-23T05:34:12.787Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5533worker2 # [ 133.173364] nix-grpc-daemon[1221]: time=2026-09-23T05:34:12.787Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5534lb # [ 133.192602] niks3-server[1137]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5535lb # [ 133.206078] niks3-server[1137]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5536worker2 # [ 133.223392] nix-grpc-daemon[1221]: time=2026-09-23T05:34:12.839Z level=INFO msg="Uploading 1 narinfos"5537lb # [ 133.217079] niks3-server[1137]: 2026/09/23 05:34:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5538lb # [ 133.227252] niks3-server[1137]: 2026/09/23 05:34:12 INFO Signed narinfos id=48 count=15539lb # [ 133.232641] niks3-server[1137]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5540worker2 # [ 133.250613] nix-grpc-daemon[1221]: time=2026-09-23T05:34:12.866Z level=INFO msg="Upload complete. (239ms)"5541lb # [ 133.244793] niks3-server[1137]: 2026/09/23 05:34:12 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5542lb # [ 133.252279] niks3-server[1137]: 2026/09/23 05:34:12 INFO Completed upload id=485543lb # [ 133.255072] niks3-server[1137]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5544worker2 # [ 133.263552] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15545worker2 # [ 133.281402] nix-daemon[839]: reaped child process 1334, status = succeeded5546client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.65 seconds)5547client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5548client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5549worker1 # [ 133.645062] systemd[1]: Listening on Nix worker-protocol over gRPC.5550worker1 # [ 133.653568] systemd[1]: Starting Nix worker-protocol over gRPC...5551worker1 # [ 133.803408] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5552worker1 # [ 133.808545] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5553worker1 # [ 133.810445] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15554worker1 # [ 133.836485] nix-grpc-daemon[1511]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5555lb: must succeed: curl -sf localhost:9901/clusters5556worker1 # [ 133.837958] nix-grpc-daemon[1511]: E0000 00:00:1790141653.418541 1511 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5557worker1 # [ 133.861436] nix-grpc-daemon[1511]: E0000 00:00:1790141653.420524 1511 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5558worker1 # [ 133.864268] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5559worker1 # [ 133.866081] nix-grpc-daemon[1511]: E0000 00:00:1790141653.427180 1551 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5560worker1 # [ 133.868472] systemd[1]: Started Nix worker-protocol over gRPC.5561worker1 # [ 133.872163] nix-grpc-daemon[1511]: E0000 00:00:1790141653.448761 1551 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5562worker1 # [ 133.876585] nix-grpc-daemon[1511]: E0000 00:00:1790141653.448916 1551 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5563worker1 # [ 133.879009] nix-grpc-daemon[1511]: E0000 00:00:1790141653.464367 1551 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5564worker2 # [ 133.807325] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:499185565worker1 # [ 133.881384] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:13Z level=info event=scheduler_connected addr=lb:500515566worker2 # [ 133.814002] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05567lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5568worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5569worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5570lb: must succeed: curl -sf localhost:9901/clusters5571lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5572lb: must succeed: curl -sf localhost:9901/clusters5573lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5574worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5575worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5576(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.82 seconds)5577subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5578client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5579client # [ 134.163378] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5580client # Running as unit: rs.service; invocation ID: 451f01f512af46969b0e02fe37f408055581client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5582worker2 # [ 134.361438] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:451405583client # [ 134.361885] nix[1406]: don't know how to build these paths:5584client # [ 134.363071] nix[1406]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5585worker2 # [ 134.380788] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05586worker2 # [ 134.387825] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:13Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15587worker2 # [ 134.427242] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:451405588worker2 # [ 134.443283] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:451405589client # [ 134.451211] nix[1406]: copying 1 paths to 'lb:50051'...5590worker2 # [ 134.458288] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:451405591client # [ 134.453900] nix[1406]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5592worker2 # [ 134.473181] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5593worker2 # [ 134.631407] nix-daemon[839]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)5594worker2 # [ 134.653591] nix-daemon[839]: reaped child process 1371, status = succeeded5595lb # [ 134.645596] niks3-server[1137]: 2026/09/23 05:34:14 INFO Received uploads request method=POST path=/api/pending_closures5596worker2 # [ 134.662473] nix-grpc-daemon[1221]: time=2026-09-23T05:34:14.278Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5597worker2 # [ 134.663959] nix-grpc-daemon[1221]: time=2026-09-23T05:34:14.278Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5598lb # [ 134.680736] niks3-server[1137]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5599worker2 # [ 134.699096] nix-grpc-daemon[1221]: time=2026-09-23T05:34:14.315Z level=INFO msg="Uploading 1 narinfos"5600lb # [ 134.693732] niks3-server[1137]: 2026/09/23 05:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5601lb # [ 134.703780] niks3-server[1137]: 2026/09/23 05:34:14 INFO Signed narinfos id=49 count=15602worker2 # [ 134.718938] nix-grpc-daemon[1221]: time=2026-09-23T05:34:14.335Z level=INFO msg="Upload complete. (230ms)"5603lb # [ 134.713211] niks3-server[1137]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5604lb # [ 134.721899] niks3-server[1137]: 2026/09/23 05:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5605worker2 # [ 134.727104] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=0 paths=1 nar_bytes_in=4165606lb # [ 134.725563] niks3-server[1137]: 2026/09/23 05:34:14 INFO Completed upload id=495607lb # [ 134.728119] niks3-server[1137]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5608worker2 # [ 134.742583] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:451405609worker2 # [ 134.748670] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5610worker2 # [ 134.753606] nix-daemon[839]: reaped child process 1366, status = succeeded5611client # [ 134.755923] nix[1406]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5612worker2 # [ 134.766082] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5613lb: must succeed: curl -sf localhost:9901/clusters5614lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5615lb: must succeed: curl -sf localhost:9901/clusters5616lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5617lb: must succeed: curl -sf localhost:9901/clusters5618lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5619lb: must succeed: curl -sf localhost:9901/clusters5620lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5621worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5622worker1 # [ 137.075716] systemd-journald[270]: Received client request to rotate journal, rotating.5623worker1 # [ 137.096608] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/335b9ae880604d8189dd723b100c300d.5624worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5625worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5626worker2 # [ 137.073678] systemd[1]: Stopping Nix worker-protocol over gRPC...5627worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5628client: must succeed: sleep 5.05629worker2 # [ 137.941912] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:17Z level=info event=draining5630lb # [ 139.618927] envoy[904]: conn peer=[2001:db8:1::1]:36738 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9658 tx=4422 ms=54045631client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5632worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5633worker2 # [ 142.660609] nix-daemon[839]: reaped child process 1375, status = succeeded5634worker2 # [ 142.752957] nix-daemon[839]: accepted connection from pid 1416, user nix-grpc-daemon (trusted)5635worker2 # [ 142.772998] nix-daemon[839]: reaped child process 1419, status = succeeded5636lb # [ 142.770874] niks3-server[1137]: 2026/09/23 05:34:22 INFO Received uploads request method=POST path=/api/pending_closures5637worker2 # [ 142.791901] nix-grpc-daemon[1221]: time=2026-09-23T05:34:22.405Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5638worker2 # [ 142.794103] nix-grpc-daemon[1221]: time=2026-09-23T05:34:22.405Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5639worker2 # [ 142.836655] nix-grpc-daemon[1221]: time=2026-09-23T05:34:22.452Z level=INFO msg="Uploading 1 narinfos"5640lb # [ 142.822085] niks3-server[1137]: 2026/09/23 05:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5641lb # [ 142.836070] niks3-server[1137]: 2026/09/23 05:34:22 INFO Signed narinfos id=50 count=15642lb # [ 142.841081] niks3-server[1137]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5643lb # [ 142.849757] niks3-server[1137]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5644lb # [ 142.858811] niks3-server[1137]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5645lb # [ 142.870981] niks3-server[1137]: 2026/09/23 05:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5646lb # [ 142.876364] niks3-server[1137]: 2026/09/23 05:34:22 INFO Completed upload id=505647worker2 # [ 142.881826] nix-grpc-daemon[1221]: time=2026-09-23T05:34:22.498Z level=INFO msg="Upload complete. (301ms)"5648lb # [ 142.878918] niks3-server[1137]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5649worker2 # [ 142.897231] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45140 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15650worker2 # [ 142.915124] nix-daemon[839]: reaped child process 1373, status = succeeded5651client # [ 142.920331] systemd[1]: rs.service: Deactivated successfully.5652client # [ 142.925335] systemd[1]: rs.service: Consumed 188ms CPU time over 8.758s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.5653worker2 # [ 142.942506] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:22Z level=info event=signal_exit5654lb # [ 143.072317] niks3-server[1137]: 2026/09/23 05:34:22 INFO lead: released remote=192.168.1.4:581725655lb # [ 143.747469] niks3-server[1137]: 2026/09/23 05:34:23 INFO lead: acquired remote=192.168.1.3:480885656worker1 # [ 143.825633] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:23Z level=info event=scheduler_take_over5657worker1 # [ 144.152309] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:23Z level=info event=scheduler_restarting addr=lb:500515658worker2 # [ 144.082728] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=scheduler_restarting5659lb # [ 144.074955] envoy[904]: [2026-09-23 05:34:23.675][904][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5660worker2 # [ 144.087733] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=scheduler_restarting addr=lb:500515661worker2 # [ 144.096805] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:499185662worker2 # [ 144.101466] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=worker_gone addr=192.168.1.3:500515663worker2 # [ 144.106019] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:499185664worker2 # [ 144.111237] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=worker_gone addr=192.168.1.4:500515665worker2 # [ 144.134806] nix-daemon[839]: reaped child process 1213, status = succeeded5666worker1 # [ 144.353291] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:23Z level=info event=scheduler_connected addr=lb:500515667worker1 # [ 144.358477] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5668worker2 # [ 144.308645] nix-grpc-daemon[1160]: ts=2026-09-23T05:34:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5669worker1 # [ 144.559102] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:24Z level=info event=scheduler_connected addr=lb:500515670worker1 # [ 144.568158] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:518605671worker1 # [ 144.573299] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05672worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.19 seconds)5673client: waiting for success: ! systemctl is-active rs5674client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5675client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5676client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5677client: must fail: journalctl -u rs | grep -q 'reconnecting'5678client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5679lb: must succeed: curl -sf localhost:9901/clusters5680lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5681worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5682worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5683worker2 # [ 144.999733] nix-grpc-daemon[1160]: E0000 00:00:1790141664.615883 1206 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 23, started true, shutdown true5684worker2 # [ 145.076873] nix-grpc-daemon[1160]: E0000 00:00:1790141664.693162 1206 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5685lb: must succeed: curl -sf localhost:9901/clusters5686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5687worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5688worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5689worker2 # [ 145.976284] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5690worker2 # [ 145.982872] systemd[1]: Stopped Nix worker-protocol over gRPC.5691worker2 # [ 145.987325] systemd[1]: nix-grpc-daemon.service: Consumed 1.822s CPU time over 37.204s wall clock time, 40.2M memory peak, 62.3K incoming IP traffic, 146.5K outgoing IP traffic.5692worker2 # [ 146.007350] systemd[1]: Starting scheduler-token.service...5693worker2 # [ 146.084576] systemd[1]: scheduler-token.service: Deactivated successfully.5694worker1 # [ 146.153672] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:25Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:518665695worker2 # [ 146.088987] systemd[1]: Finished scheduler-token.service.5696worker2 # [ 146.090880] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 79ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5697worker2 # [ 146.116316] systemd[1]: Starting Nix worker-protocol over gRPC...5698worker2 # [ 146.224358] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5699worker2 # [ 146.226329] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5700worker2 # [ 146.230133] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15701worker2 # [ 146.236062] nix-grpc-daemon[1429]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5702worker2 # [ 146.237544] nix-grpc-daemon[1429]: E0000 00:00:1790141665.850968 1429 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5703worker2 # [ 146.246680] nix-grpc-daemon[1429]: E0000 00:00:1790141665.857361 1429 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5704worker2 # [ 146.280126] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5705worker2 # [ 146.281853] systemd[1]: Started Nix worker-protocol over gRPC.5706worker2 # [ 146.282658] nix-grpc-daemon[1429]: E0000 00:00:1790141665.866320 1470 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5707worker2 # [ 146.285478] nix-grpc-daemon[1429]: E0000 00:00:1790141665.881512 1470 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5708worker2 # [ 146.288348] nix-grpc-daemon[1429]: E0000 00:00:1790141665.881670 1470 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5709worker2 # [ 146.290859] nix-grpc-daemon[1429]: E0000 00:00:1790141665.896004 1470 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5710worker2 # [ 146.298963] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:25Z level=info event=scheduler_connected addr=lb:500515711worker1 # [ 146.378814] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:25Z level=info event=oidc_jwks provider=mock keys=15712worker1 # [ 146.383962] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:518605713worker1 # [ 146.393267] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05714lb: must succeed: curl -sf localhost:9901/clusters5715lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5716worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5717worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5718(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.00 seconds)5719subtest: scheduler restart mid-build: build finishes, no double build5720client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5721client # [ 147.164936] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5722client # Running as unit: mid.service; invocation ID: 483d48691df540d494ba2e008b5669305723client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5724worker2 # [ 147.370917] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505665725worker2 # [ 147.385358] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5726client # [ 147.394052] nix[1442]: don't know how to build these paths:5727client # [ 147.395299] nix[1442]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5728worker1 # [ 147.485210] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:27Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05729worker1 # [ 147.491401] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:27Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15730worker1 # [ 147.551950] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518705731worker1 # [ 147.566465] nix-daemon[854]: accepted connection from pid 1511, user nix-grpc-daemon (trusted)5732worker1 # [ 147.589377] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5)5733worker1 # [ 147.675213] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5)5734worker1 # [ 147.836483] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 357 ms (attempt 3/5)5735worker1 # [ 148.203006] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 614 ms (attempt 4/5)5736worker1 # [ 148.820603] nix-grpc-daemon[1511]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5737worker1 # [ 148.828570] nix-grpc-daemon[1511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5738worker1 # [ 148.846360] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:518705739client # [ 148.783791] nix[1442]: copying 1 paths to 'lb:50051'...5740worker1 # [ 148.864531] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:518705741client # [ 148.790484] nix[1442]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5742worker1 # [ 148.879469] nix-daemon[854]: accepted connection from pid 1511, user nix-grpc-daemon (trusted)5743worker1 # [ 148.905231] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=info event=niks3_push_started pid=16415744worker1 # [ 149.049776] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.653Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5745worker1 # [ 149.158562] nix-daemon[854]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)5746worker1 # [ 149.175399] nix-daemon[854]: reaped child process 1656, status = succeeded5747lb # [ 149.106239] niks3-server[1137]: 2026/09/23 05:34:28 INFO Received uploads request method=POST path=/api/pending_closures5748worker1 # [ 149.194006] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.796Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5749worker1 # [ 149.195444] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.796Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5750lb # [ 149.148936] niks3-server[1137]: 2026/09/23 05:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5751worker1 # [ 149.236508] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.839Z level=INFO msg="Uploading 1 narinfos"5752lb # [ 149.163096] niks3-server[1137]: 2026/09/23 05:34:28 INFO Signed narinfos id=51 count=15753lb # [ 149.171519] niks3-server[1137]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5754lb # [ 149.187891] niks3-server[1137]: 2026/09/23 05:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5755lb # [ 149.199254] niks3-server[1137]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5756lb # [ 149.206053] niks3-server[1137]: 2026/09/23 05:34:28 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5757lb # [ 149.210764] niks3-server[1137]: 2026/09/23 05:34:28 INFO Completed upload id=515758worker1 # [ 149.287572] nix-grpc-daemon[1641]: time=2026-09-23T05:34:28.891Z level=INFO msg="Upload complete. (234ms)"5759worker1 # [ 149.298860] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51870 duration_s=0 paths=1 nar_bytes_in=4245760worker1 # [ 149.316265] nix-grpc-daemon[1511]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:518705761worker1 # [ 149.327263] nix-daemon[854]: accepted connection from pid 1511, user nix-grpc-daemon (trusted)5762worker1 # [ 149.330853] nix-daemon[854]: reaped child process 1639, status = succeeded5763client # [ 149.265850] nix[1442]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5764worker1 # [ 149.344869] nix-daemon[854]: accepted connection from pid 1511, user nix-grpc-daemon (trusted)5765lb # [ 149.623938] envoy[904]: conn peer=[2001:db8:1::1]:47100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1518 ms=85465766lb # [ 149.629879] envoy[904]: conn peer=[2001:db8:1::4]:59060 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=27953 tx=1774 ms=353185767lb: must succeed: curl -sf localhost:9901/clusters5768lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5769lb: must succeed: curl -sf localhost:9901/clusters5770lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5771worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5772worker1 # [ 151.101547] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1511 (nix-grpc-daemon) on client request.5773worker1 # [ 151.103131] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5774lb # [ 151.041496] niks3-server[1137]: 2026/09/23 05:34:30 INFO lead: released remote=192.168.1.3:480885775client # [ 151.048813] nix[1442]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5776worker2 # [ 151.055550] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:30Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5777worker1 # [ 151.132458] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5778worker1 # [ 151.133967] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5779lb # [ 151.050314] envoy[904]: [2026-09-23 05:34:30.651][904][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5780client # [ 151.055143] nix[1442]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5781worker1 # [ 151.135234] systemd[1]: nix-grpc-daemon.service: Consumed 529ms CPU time over 17.473s wall clock time, 34.7M memory peak, 15.6K incoming IP traffic, 26.7K outgoing IP traffic.5782worker1 # [ 151.157497] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5783worker1 # [ 151.159014] systemd[1]: Starting Nix worker-protocol over gRPC...5784worker1 # [ 151.196195] nix-daemon[854]: reaped child process 1629, status = succeeded5785worker1 # [ 151.199615] nix-daemon[854]: reaped child process 1658, status = succeeded5786worker1 # [ 151.208661] nix-daemon[1660]: unexpected Nix daemon error: error: interrupted by the user5787worker1 # [ 151.213973] nix-daemon[854]: reaped child process 1660, status = failed with exit code 15788worker1 # [ 151.274237] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5789worker1 # [ 151.276757] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5790worker1 # [ 151.280441] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15791worker1 # [ 151.285591] nix-grpc-daemon[1696]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5792worker1 # [ 151.287030] nix-grpc-daemon[1696]: E0000 00:00:1790141670.888823 1696 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5793lb # [ 151.225372] niks3-server[1137]: 2026/09/23 05:34:30 INFO lead: acquired remote=192.168.1.3:509605794worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.24 seconds)5795client: must succeed: sleep 5.05796worker1 # [ 151.320429] nix-grpc-daemon[1696]: E0000 00:00:1790141670.894666 1696 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5797worker1 # [ 151.323962] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5798worker2 # [ 151.255990] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:30Z level=info event=scheduler_connected addr=lb:500515799worker1 # [ 151.347333] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=scheduler_take_over5800worker1 # [ 151.348946] nix-grpc-daemon[1696]: E0000 00:00:1790141670.906513 1737 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5801worker1 # [ 151.351437] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:477725802worker1 # [ 151.353023] systemd[1]: Started Nix worker-protocol over gRPC.5803worker1 # [ 151.353840] nix-grpc-daemon[1696]: E0000 00:00:1790141670.930627 1737 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5804worker1 # [ 151.358782] nix-grpc-daemon[1696]: E0000 00:00:1790141670.930756 1737 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5805worker1 # [ 151.363385] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=oidc_jwks provider=mock keys=15806worker1 # [ 151.365180] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:477885807worker1 # [ 151.370804] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05808worker1 # [ 151.372736] nix-grpc-daemon[1696]: E0000 00:00:1790141670.950614 1737 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5809worker1 # [ 151.375075] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=scheduler_connected addr=lb:500515810worker1 # [ 151.377244] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:477885811worker1 # [ 151.379386] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05812worker1 # [ 151.638177] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:31Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05813worker1 # [ 151.644062] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:31Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15814worker2 # [ 151.629592] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574045815worker2 # [ 151.644959] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)5816worker2 # [ 151.743214] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)5817worker2 # [ 151.753822] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 275 ms (attempt 3/5)5818worker2 # [ 152.037867] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 4/5)5819worker2 # [ 152.122254] nix-grpc-daemon[1429]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5820worker2 # [ 152.128450] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5821worker2 # [ 152.143765] nix-grpc-daemon[1429]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5822worker2 # [ 152.167800] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5823worker2 # [ 152.190293] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5824client # [ 152.182719] nix[1442]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5825client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5826client: waiting for success: ! systemctl is-active mid5827worker2 # [ 156.616966] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:36Z level=info event=niks3_push_started pid=15365828worker2 # [ 156.634045] nix-daemon[839]: reaped child process 1511, status = succeeded5829worker2 # [ 156.770252] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.386Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5830worker2 # [ 156.877480] nix-daemon[839]: accepted connection from pid 1549, user nix-grpc-daemon (trusted)5831worker2 # [ 156.898971] nix-daemon[839]: reaped child process 1552, status = succeeded5832lb # [ 156.894105] niks3-server[1137]: 2026/09/23 05:34:36 INFO Received uploads request method=POST path=/api/pending_closures5833worker2 # [ 156.916058] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.531Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5834worker2 # [ 156.917556] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.531Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5835lb # [ 156.953017] niks3-server[1137]: 2026/09/23 05:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5836worker2 # [ 156.973889] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.588Z level=INFO msg="Uploading 1 narinfos"5837lb # [ 156.968353] niks3-server[1137]: 2026/09/23 05:34:36 INFO Signed narinfos id=52 count=15838lb # [ 156.980239] niks3-server[1137]: 2026/09/23 05:34:36 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5839lb # [ 156.983178] niks3-server[1137]: 2026/09/23 05:34:36 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5840lb # [ 157.002747] niks3-server[1137]: 2026/09/23 05:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5841lb # [ 157.005557] niks3-server[1137]: 2026/09/23 05:34:36 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5842lb # [ 157.009678] niks3-server[1137]: 2026/09/23 05:34:36 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5843lb # [ 157.014128] niks3-server[1137]: 2026/09/23 05:34:36 INFO Completed upload id=525844worker2 # [ 157.020963] nix-grpc-daemon[1536]: time=2026-09-23T05:34:36.637Z level=INFO msg="Upload complete. (247ms)"5845worker2 # [ 157.032205] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57404 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15846worker2 # [ 157.052328] nix-daemon[839]: reaped child process 1509, status = succeeded5847client # [ 157.062042] systemd[1]: mid.service: Deactivated successfully.5848client # [ 157.067154] systemd[1]: mid.service: Consumed 197ms CPU time over 9.898s wall clock time, 41.3M memory peak, 8.7K incoming IP traffic, 10.3K outgoing IP traffic.5849client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5850client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5851client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5852(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.60 seconds)5853subtest: interrupting the client stops the build on the worker5854client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5855client # [ 157.736545] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5856client # Running as unit: intr.service; invocation ID: 97153613141c4c558e48483ed1a088a65857client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5858worker2 # [ 157.915591] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:574045859client # [ 157.915518] nix[1478]: don't know how to build these paths:5860client # [ 157.916531] nix[1478]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5861worker1 # [ 158.005369] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:37Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05862worker1 # [ 158.011758] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:37Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15863worker1 # [ 158.072125] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478165864worker1 # [ 158.086407] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)5865worker1 # [ 158.109545] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 1/5)5866worker1 # [ 158.165478] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 2/5)5867worker1 # [ 158.235075] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 3/5)5868worker1 # [ 158.253855] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 625 ms (attempt 4/5)5869worker1 # [ 158.918750] nix-grpc-daemon[1696]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5870worker1 # [ 158.919068] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5871worker1 # [ 158.943492] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478165872client # [ 158.880336] nix[1478]: copying 1 paths to 'lb:50051'...5873worker1 # [ 158.958858] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478165874client # [ 158.884173] nix[1478]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5875worker1 # [ 158.975498] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)5876worker1 # [ 159.018429] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:38Z level=info event=niks3_push_started pid=17775877worker1 # [ 159.171434] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.774Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5878worker1 # [ 159.276895] nix-daemon[854]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)5879worker1 # [ 159.296136] nix-daemon[854]: reaped child process 1794, status = succeeded5880lb # [ 159.222858] niks3-server[1137]: 2026/09/23 05:34:38 INFO Received uploads request method=POST path=/api/pending_closures5881worker1 # [ 159.311713] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.914Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5882worker1 # [ 159.315016] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.914Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5883lb # [ 159.261974] niks3-server[1137]: 2026/09/23 05:34:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign5884worker1 # [ 159.351398] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.954Z level=INFO msg="Uploading 1 narinfos"5885lb # [ 159.278684] niks3-server[1137]: 2026/09/23 05:34:38 INFO Signed narinfos id=53 count=15886lb # [ 159.286170] niks3-server[1137]: 2026/09/23 05:34:38 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5887lb # [ 159.294234] niks3-server[1137]: 2026/09/23 05:34:38 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5888worker1 # [ 159.385383] nix-grpc-daemon[1777]: time=2026-09-23T05:34:38.989Z level=INFO msg="Upload complete. (210ms)"5889lb # [ 159.308307] niks3-server[1137]: 2026/09/23 05:34:38 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete5890lb # [ 159.314769] niks3-server[1137]: 2026/09/23 05:34:38 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5891worker1 # [ 159.392247] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=4245892lb # [ 159.319563] niks3-server[1137]: 2026/09/23 05:34:38 INFO Completed upload id=535893worker1 # [ 159.407181] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478165894worker1 # [ 159.413212] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)5895worker1 # [ 159.418037] nix-daemon[854]: reaped child process 1772, status = succeeded5896worker1 # [ 159.429703] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)5897client # [ 159.350882] nix[1478]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5898lb # [ 159.624243] envoy[904]: conn peer=[2001:db8:1::3]:33648 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=831 tx=393 ms=172585899lb # [ 159.629208] envoy[904]: conn peer=[2001:db8:1::1]:40846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1988 ms=96845900client: must succeed: systemctl kill -s INT intr5901worker1 # [ 160.394259] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:39Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5902client # [ 160.317577] systemd[1]: intr.service: Sent signal SIGINT to main process 1478 (nix) on client request.5903client # [ 160.325771] nix[1478]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5904client # [ 160.335398] nix[1478]: error: interrupted by the user5905client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5906client # [ 160.357905] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5907client # [ 160.362334] systemd[1]: intr.service: Failed with result 'exit-code'.5908client # [ 160.364436] systemd[1]: intr.service: Consumed 164ms CPU time over 2.624s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8K outgoing IP traffic.5909worker1 # [ 160.474857] nix-daemon[1798]: unexpected Nix daemon error: error: interrupted by the user5910worker1 # [ 160.479966] nix-daemon[854]: reaped child process 1796, status = succeeded5911worker1 # [ 160.484297] nix-daemon[854]: reaped child process 1798, status = failed with exit code 15912(finished: subtest: interrupting the client stops the build on the worker, in 2.77 seconds)5913subtest: a deploy mid-build drains: the build finishes, the new generation takes over5914client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5915client # [ 160.535141] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5916client # Running as unit: sw.service; invocation ID: 89c55c05c4ea43f68fd916032b5589005917client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)5918client # [ 160.727390] nix[1499]: don't know how to build these paths:5919client # [ 160.728613] nix[1499]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5920worker1 # [ 160.801855] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478165921worker1 # [ 160.817981] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:40Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05922worker1 # [ 160.823721] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:40Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15923worker2 # [ 160.796610] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574045924worker2 # [ 160.813164] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:574045925client # [ 160.820141] nix[1499]: copying 1 paths to 'lb:50051'...5926client # [ 160.822424] nix[1499]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5927worker2 # [ 160.828623] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:574045928worker2 # [ 160.844998] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5929worker2 # [ 161.008938] nix-daemon[839]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5930worker2 # [ 161.028499] nix-daemon[839]: reaped child process 1579, status = succeeded5931lb # [ 161.027747] niks3-server[1137]: 2026/09/23 05:34:40 INFO Received uploads request method=POST path=/api/pending_closures5932worker2 # [ 161.046582] nix-grpc-daemon[1536]: time=2026-09-23T05:34:40.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5933worker2 # [ 161.048455] nix-grpc-daemon[1536]: time=2026-09-23T05:34:40.661Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5934lb # [ 161.056437] niks3-server[1137]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5935worker2 # [ 161.077307] nix-grpc-daemon[1536]: time=2026-09-23T05:34:40.692Z level=INFO msg="Uploading 1 narinfos"5936lb # [ 161.070141] niks3-server[1137]: 2026/09/23 05:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign5937lb # [ 161.075931] niks3-server[1137]: 2026/09/23 05:34:40 INFO Signed narinfos id=54 count=15938lb # [ 161.082566] niks3-server[1137]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5939lb # [ 161.092506] niks3-server[1137]: 2026/09/23 05:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete5940lb # [ 161.101808] niks3-server[1137]: 2026/09/23 05:34:40 INFO Completed upload id=545941worker2 # [ 161.108636] nix-grpc-daemon[1536]: time=2026-09-23T05:34:40.725Z level=INFO msg="Upload complete. (247ms)"5942lb # [ 161.106190] niks3-server[1137]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5943worker2 # [ 161.120938] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57404 duration_s=0 paths=1 nar_bytes_in=4165944worker2 # [ 161.131750] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574045945worker2 # [ 161.141569] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5946worker2 # [ 161.144353] nix-daemon[839]: reaped child process 1574, status = succeeded5947client # [ 161.147871] nix[1499]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5948worker2 # [ 161.158639] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5949worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25950worker2 # Checking switch inhibitors... done5951worker2 # [ 163.546396] nixos[1615]: switching to system configuration /nix/store/n3w0z58pg0if7vb7i025y1idh9rrsqkh-nixos-system-worker2-test5952worker2 # activating the configuration...5953worker2 # [ 163.551714] systemd[1]: Stopped target Remote File Systems.5954worker2 # [ 163.557211] systemd[1]: Stopped target Local File Systems.5955worker2 # setting up /etc...5956worker2 # [ 164.124801] systemd[1]: Reload requested from client PID 1615 ('.switch-to-conf') (unit backdoor.service)...5957worker2 # [ 164.126924] systemd[1]: Reloading...5958worker2 # [ 169.019904] systemd[1]: Reloading finished in 4886 ms.5959worker2 # restarting sysinit-reactivation.target5960worker2 # [ 169.136756] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5961worker2 # [ 169.258180] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5962worker2 # [ 169.261123] systemd[1]: Reached target Reactivate sysinit units.5963worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5964worker2 # [ 169.265372] dbus-broker-launch[493]: Caught SIGHUP, trigger reload.5965worker2 # [ 169.266337] systemd[1]: Reloading D-Bus System Message Bus...5966worker2 # [ 169.273957] systemd[1]: Reloading Nix worker-protocol over gRPC...5967worker2 # [ 169.291682] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'...5968worker2 # [ 169.301331] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync'5969worker2 # [ 169.302331] dbus-broker-launch[493]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5970worker2 # [ 169.316226] systemd[1]: Reloaded Nix worker-protocol over gRPC.5971worker2 # [ 169.330850] systemd[1]: Reloaded D-Bus System Message Bus.5972worker2 # [ 169.338944] systemd[1]: Reached target Remote File Systems.5973worker2 # [ 169.342116] systemd[1]: Starting linger-users.service...5974worker2 # [ 169.348411] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5975worker2 # [ 169.349782] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5976worker2 # [ 169.360237] systemd[1]: Reached target Local File Systems.5977worker2 # [ 169.372321] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5978worker2 # [ 169.373752] systemd[1]: Starting Create SUID/SGID Wrappers...5979worker2 # [ 169.374530] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5980worker2 # [ 169.375795] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675981worker2 # [ 169.389996] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5982worker2 # [ 169.391634] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5983worker2 # [ 169.403807] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5984worker2 # [ 169.421298] systemd[1]: linger-users.service: Deactivated successfully.5985worker2 # [ 169.424757] systemd[1]: Finished linger-users.service.5986lb # [ 169.628937] envoy[904]: conn peer=[2001:db8:1::1]:52020 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=809 ms=24355987worker2 # [ 169.718435] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5988worker2 # [ 169.719652] systemd[1]: Finished Create SUID/SGID Wrappers.5989worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5990worker2 # [ 169.975215] nixos[1615]: finished switching to system configuration /nix/store/n3w0z58pg0if7vb7i025y1idh9rrsqkh-nixos-system-worker2-test5991worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.88 seconds)5992worker2: must succeed: pgrep -f 'read -t [9]0 x'5993worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)5994client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&25995worker2 # [ 170.259512] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:49Z level=info event=draining5996worker1 # [ 170.333479] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478165997client # don't know how to build these paths:5998client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5999worker1 # [ 170.346788] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166000worker1 # [ 170.373414] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:49Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06001worker1 # [ 170.380064] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:49Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16002worker1 # [ 170.426241] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166003worker1 # [ 170.442204] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166004client # copying 1 paths to 'lb:50051'...6005worker1 # [ 170.458103] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166006client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6007worker1 # [ 170.481421] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6008worker1 # [ 170.639295] nix-daemon[854]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)6009worker1 # [ 170.661636] nix-daemon[854]: reaped child process 1848, status = succeeded6010lb # [ 170.583530] niks3-server[1137]: 2026/09/23 05:34:50 INFO Received uploads request method=POST path=/api/pending_closures6011worker1 # [ 170.671769] nix-grpc-daemon[1777]: time=2026-09-23T05:34:50.274Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6012worker1 # [ 170.673928] nix-grpc-daemon[1777]: time=2026-09-23T05:34:50.274Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6013lb # [ 170.624961] niks3-server[1137]: 2026/09/23 05:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6014lb # [ 170.637938] niks3-server[1137]: 2026/09/23 05:34:50 INFO Signed narinfos id=55 count=16015worker1 # [ 170.718835] nix-grpc-daemon[1777]: time=2026-09-23T05:34:50.322Z level=INFO msg="Uploading 1 narinfos"6016lb # [ 170.652624] niks3-server[1137]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6017lb # [ 170.663507] niks3-server[1137]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6018lb # [ 170.671078] niks3-server[1137]: 2026/09/23 05:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6019lb # [ 170.677004] niks3-server[1137]: 2026/09/23 05:34:50 INFO Completed upload id=556020worker1 # [ 170.753210] nix-grpc-daemon[1777]: time=2026-09-23T05:34:50.357Z level=INFO msg="Upload complete. (254ms)"6021lb # [ 170.679781] niks3-server[1137]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6022worker1 # [ 170.764382] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=5286023worker1 # [ 170.783726] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166024worker1 # [ 170.791640] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6025worker1 # [ 170.798503] nix-daemon[854]: reaped child process 1843, status = succeeded6026client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6027worker1 # [ 170.813130] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6028client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6029client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6030client # farm-a> LOG-farm-a-during-drain6031worker1 # [ 172.324629] nix-daemon[854]: reaped child process 1852, status = succeeded6032worker1 # [ 172.397586] nix-daemon[854]: accepted connection from pid 1868, user nix-grpc-daemon (trusted)6033worker1 # [ 172.418508] nix-daemon[854]: reaped child process 1871, status = succeeded6034lb # [ 172.346919] niks3-server[1137]: 2026/09/23 05:34:51 INFO Received uploads request method=POST path=/api/pending_closures6035worker1 # [ 172.434924] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.036Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6036worker1 # [ 172.437924] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.036Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6037lb # [ 172.382486] niks3-server[1137]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6038lb # [ 172.399633] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6039worker1 # [ 172.487376] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.091Z level=INFO msg="Uploading 1 narinfos"6040lb # [ 172.412282] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6041lb # [ 172.422374] niks3-server[1137]: 2026/09/23 05:34:52 INFO Signed narinfos id=56 count=16042lb # [ 172.431816] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6043worker1 # [ 172.514625] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.118Z level=INFO msg="Upload complete. (286ms)"6044lb # [ 172.439049] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6045lb # [ 172.443821] niks3-server[1137]: 2026/09/23 05:34:52 INFO Completed upload id=566046lb # [ 172.450057] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6047worker1 # [ 172.529785] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16048worker1 # [ 172.553228] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06049worker1 # [ 172.557706] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16050worker1 # [ 172.562272] nix-daemon[854]: reaped child process 1850, status = succeeded6051worker1 # [ 172.589627] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166052worker1 # [ 172.611406] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166053worker1 # [ 172.625681] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.229Z level=INFO msg="All 1 paths already cached"6054client # copying 1 paths to 'lb:50051'...6055worker1 # [ 172.636377] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166056client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6057worker1 # [ 172.658954] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6058worker1 # [ 172.818043] nix-daemon[854]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)6059worker1 # [ 172.833741] nix-daemon[854]: reaped child process 1880, status = succeeded6060lb # [ 172.763630] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received uploads request method=POST path=/api/pending_closures6061worker1 # [ 172.849146] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.452Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6062worker1 # [ 172.850731] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.452Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6063lb # [ 172.793294] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6064worker1 # [ 172.880691] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.484Z level=INFO msg="Uploading 1 narinfos"6065lb # [ 172.808804] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6066lb # [ 172.819488] niks3-server[1137]: 2026/09/23 05:34:52 INFO Signed narinfos id=57 count=16067lb # [ 172.825961] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6068worker1 # [ 172.912323] nix-grpc-daemon[1777]: time=2026-09-23T05:34:52.516Z level=INFO msg="Upload complete. (236ms)"6069lb # [ 172.836804] niks3-server[1137]: 2026/09/23 05:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6070lb # [ 172.840459] niks3-server[1137]: 2026/09/23 05:34:52 INFO Completed upload id=576071lb # [ 172.844451] niks3-server[1137]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6072worker1 # [ 172.924315] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=6406073worker1 # [ 172.938152] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166074worker1 # [ 172.947525] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6075worker1 # [ 172.950965] nix-daemon[854]: reaped child process 1875, status = succeeded6076client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6077worker1 # [ 172.964159] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6078client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6079client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6080client # farm-b> LOG-farm-b-during-drain6081worker1 # [ 173.822895] nix-daemon[854]: reaped child process 1884, status = succeeded6082worker1 # [ 173.949694] nix-daemon[854]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)6083worker1 # [ 173.969711] nix-daemon[854]: reaped child process 1903, status = succeeded6084lb # [ 173.898604] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received uploads request method=POST path=/api/pending_closures6085worker1 # [ 173.989840] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.592Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6086worker1 # [ 173.991828] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.592Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6087lb # [ 173.945406] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6088worker1 # [ 174.031347] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.634Z level=INFO msg="Uploading 1 narinfos"6089lb # [ 173.960582] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6090lb # [ 173.972999] niks3-server[1137]: 2026/09/23 05:34:53 INFO Signed narinfos id=58 count=16091lb # [ 173.975709] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6092lb # [ 173.989769] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6093worker1 # [ 174.066770] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.670Z level=INFO msg="Upload complete. (293ms)"6094lb # [ 173.994672] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6095lb # [ 173.999154] niks3-server[1137]: 2026/09/23 05:34:53 INFO Completed upload id=586096lb # [ 174.002619] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6097worker1 # [ 174.082243] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16098worker1 # [ 174.104341] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06099worker1 # [ 174.110066] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16100worker1 # [ 174.115368] nix-daemon[854]: reaped child process 1882, status = succeeded6101worker1 # [ 174.145589] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166102worker1 # [ 174.160814] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166103worker1 # [ 174.174505] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.778Z level=INFO msg="All 2 paths already cached"6104client # copying 1 paths to 'lb:50051'...6105client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6106worker1 # [ 174.186069] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166107worker1 # [ 174.203071] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6108worker1 # [ 174.358153] nix-daemon[854]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)6109worker1 # [ 174.377683] nix-daemon[854]: reaped child process 1912, status = succeeded6110lb # [ 174.305767] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received uploads request method=POST path=/api/pending_closures6111worker1 # [ 174.391432] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.995Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6112worker1 # [ 174.393091] nix-grpc-daemon[1777]: time=2026-09-23T05:34:53.995Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6113lb # [ 174.335425] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6114worker1 # [ 174.420260] nix-grpc-daemon[1777]: time=2026-09-23T05:34:54.024Z level=INFO msg="Uploading 1 narinfos"6115lb # [ 174.350368] niks3-server[1137]: 2026/09/23 05:34:53 INFO Signed narinfos id=59 count=16116lb # [ 174.357870] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6117lb # [ 174.365771] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6118lb # [ 174.376115] niks3-server[1137]: 2026/09/23 05:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6119worker1 # [ 174.457413] nix-grpc-daemon[1777]: time=2026-09-23T05:34:54.061Z level=INFO msg="Upload complete. (241ms)"6120lb # [ 174.383169] niks3-server[1137]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6121lb # [ 174.388128] niks3-server[1137]: 2026/09/23 05:34:53 INFO Completed upload id=596122worker1 # [ 174.469044] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=7686123worker1 # [ 174.483799] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166124worker1 # [ 174.490317] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6125worker1 # [ 174.495362] nix-daemon[854]: reaped child process 1907, status = succeeded6126client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6127worker1 # [ 174.507902] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6128client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6129client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6130client # farm-top> LOG-farm-top-during-drain6131worker1 # [ 175.947391] nix-daemon[854]: reaped child process 1916, status = succeeded6132worker1 # [ 176.052347] nix-daemon[854]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6133worker1 # [ 176.069632] nix-daemon[854]: reaped child process 1935, status = succeeded6134lb # [ 176.000963] niks3-server[1137]: 2026/09/23 05:34:55 INFO Received uploads request method=POST path=/api/pending_closures6135worker1 # [ 176.088374] nix-grpc-daemon[1777]: time=2026-09-23T05:34:55.691Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6136worker1 # [ 176.089847] nix-grpc-daemon[1777]: time=2026-09-23T05:34:55.691Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6137lb # [ 176.038551] niks3-server[1137]: 2026/09/23 05:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6138worker1 # [ 176.124457] nix-grpc-daemon[1777]: time=2026-09-23T05:34:55.728Z level=INFO msg="Uploading 1 narinfos"6139lb # [ 176.053424] niks3-server[1137]: 2026/09/23 05:34:55 INFO Signed narinfos id=60 count=16140lb # [ 176.061525] niks3-server[1137]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6141lb # [ 176.068976] niks3-server[1137]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6142lb # [ 176.078960] niks3-server[1137]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6143worker1 # [ 176.163364] nix-grpc-daemon[1777]: time=2026-09-23T05:34:55.767Z level=INFO msg="Upload complete. (242ms)"6144lb # [ 176.086646] niks3-server[1137]: 2026/09/23 05:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6145lb # [ 176.091426] niks3-server[1137]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6146lb # [ 176.097967] niks3-server[1137]: 2026/09/23 05:34:55 INFO Completed upload id=606147worker1 # [ 176.174197] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16148worker1 # [ 176.191570] nix-daemon[854]: reaped child process 1914, status = succeeded6149client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.13 seconds)6150client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6151client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6152worker2: must succeed: pkill -f 'read -t [9]0 x'6153worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6154client: waiting for success: ! systemctl is-active sw6155worker2 # [ 176.635202] nix-daemon[839]: reaped child process 1583, status = succeeded6156worker2 # [ 176.770048] nix-daemon[839]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)6157worker2 # [ 176.790428] nix-daemon[839]: reaped child process 1758, status = succeeded6158lb # [ 176.788647] niks3-server[1137]: 2026/09/23 05:34:56 INFO Received uploads request method=POST path=/api/pending_closures6159worker2 # [ 176.806449] nix-grpc-daemon[1536]: time=2026-09-23T05:34:56.421Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6160worker2 # [ 176.808167] nix-grpc-daemon[1536]: time=2026-09-23T05:34:56.421Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6161worker2 # [ 176.849184] nix-grpc-daemon[1536]: time=2026-09-23T05:34:56.465Z level=INFO msg="Uploading 1 narinfos"6162lb # [ 176.831219] niks3-server[1137]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6163lb # [ 176.845819] niks3-server[1137]: 2026/09/23 05:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6164lb # [ 176.856982] niks3-server[1137]: 2026/09/23 05:34:56 INFO Signed narinfos id=61 count=16165lb # [ 176.859526] niks3-server[1137]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6166worker2 # [ 176.881073] nix-grpc-daemon[1536]: time=2026-09-23T05:34:56.497Z level=INFO msg="Upload complete. (294ms)"6167lb # [ 176.874109] niks3-server[1137]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6168lb # [ 176.879111] niks3-server[1137]: 2026/09/23 05:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6169lb # [ 176.885217] niks3-server[1137]: 2026/09/23 05:34:56 INFO Completed upload id=616170worker2 # [ 176.893230] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57404 duration_s=15 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16171lb # [ 176.889629] niks3-server[1137]: 2026/09/23 05:34:56 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6172worker2 # [ 176.914517] nix-daemon[839]: reaped child process 1581, status = succeeded6173client # [ 176.923950] systemd[1]: sw.service: Deactivated successfully.6174client # [ 176.928447] systemd[1]: sw.service: Consumed 184ms CPU time over 16.392s wall clock time, 41.4M memory peak, 7.5K incoming IP traffic, 8.6K outgoing IP traffic.6175worker2 # [ 177.261619] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:56Z level=info event=signal_exit6176client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6177client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6178client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6179worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6180worker2 # [ 177.621955] systemd[1]: Starting scheduler-token.service...6181worker2 # [ 177.684516] systemd[1]: scheduler-token.service: Deactivated successfully.6182worker2 # [ 177.688576] systemd[1]: Finished scheduler-token.service.6183worker2 # [ 177.690140] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 66ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6184worker2 # [ 179.259041] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:58Z level=info event=scheduler_restarting6185worker1 # [ 179.331269] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477886186worker1 # [ 179.336511] nix-grpc-daemon[1696]: ts=2026-09-23T05:34:58Z level=info event=worker_gone addr=192.168.1.4:500516187worker2 # [ 179.286382] nix-grpc-daemon[1429]: ts=2026-09-23T05:34:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6188worker2 # [ 179.313565] nix-daemon[839]: reaped child process 1486, status = succeeded6189lb # [ 179.623115] envoy[904]: conn peer=[2001:db8:1::1]:52436 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9621 tx=4414 ms=58696190lb # [ 179.628257] envoy[904]: conn peer=[2001:db8:1::1]:52032 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3047 tx=1559 ms=161806191lb # [ 179.633140] envoy[904]: conn peer=[2001:db8:1::4]:48672 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2289 tx=476 ms=329786192worker2 # [ 180.032582] nix-grpc-daemon[1429]: E0000 00:00:1790141699.648813 1475 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6193worker2 # [ 180.286241] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6194worker2 # [ 180.292406] systemd[1]: nix-grpc-daemon.service: Consumed 875ms CPU time over 34.182s wall clock time, 36.3M memory peak, 21.9K incoming IP traffic, 29.9K outgoing IP traffic.6195worker2 # [ 180.313669] systemd[1]: Starting Nix worker-protocol over gRPC...6196worker2 # [ 180.464448] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6197worker2 # [ 180.466913] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6198worker2 # [ 180.470852] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16199worker2 # [ 180.476054] nix-grpc-daemon[1775]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6200worker2 # [ 180.477705] nix-grpc-daemon[1775]: E0000 00:00:1790141700.092115 1775 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6201worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.98 seconds)6202lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6203lb: must succeed: curl -sf localhost:9901/clusters6204worker2 # [ 180.524511] nix-grpc-daemon[1775]: E0000 00:00:1790141700.098644 1775 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6205worker2 # [ 180.527280] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6206worker2 # [ 180.529274] nix-grpc-daemon[1775]: E0000 00:00:1790141700.106312 1816 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6207worker2 # [ 180.531589] nix-grpc-daemon[1775]: E0000 00:00:1790141700.122551 1816 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6208worker2 # [ 180.534060] nix-grpc-daemon[1775]: E0000 00:00:1790141700.122769 1816 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6209worker2 # [ 180.539548] nix-grpc-daemon[1775]: E0000 00:00:1790141700.139397 1816 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6210worker2 # [ 180.544097] systemd[1]: Started Nix worker-protocol over gRPC.6211worker2 # [ 180.545065] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=info event=scheduler_connected addr=lb:500516212worker1 # [ 180.617233] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:477886213worker1 # [ 180.623469] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06214lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6215lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.08 seconds)6216lb: must succeed: curl -sf localhost:9901/clusters6217lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6218worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6219worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6220(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 20.24 seconds)6221subtest: queue gauges show a burst while every build blocks6222lb: must succeed: curl -sf localhost:9901/clusters6223lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6224lb: must succeed: curl -sf localhost:9901/clusters6225lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6226worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6227worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6228client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6229client # [ 180.947802] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6230client # Running as unit: burst.service; invocation ID: 17ca131077564394ac222f4ef3b6812c6231client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6232lb: must succeed: curl -sf localhost:9901/clusters6233lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6234lb: must succeed: curl -sf localhost:9901/clusters6235lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6236worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6237worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6238worker2 # [ 181.158523] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:331006239worker2 # [ 181.173414] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6240client # [ 181.184354] nix[1555]: don't know how to build these paths:6241client # [ 181.186572] nix[1555]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6242client # [ 181.189444] nix[1555]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6243client # [ 181.191379] nix[1555]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6244client # [ 181.193200] nix[1555]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6245worker1 # [ 181.275613] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06246worker1 # [ 181.282738] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16247worker1 # [ 181.289828] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06248worker1 # [ 181.295283] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16249worker1 # [ 181.302531] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06250worker1 # [ 181.308239] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06251worker1 # [ 181.322556] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166252worker2 # [ 181.253054] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331006253worker1 # [ 181.338926] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166254worker2 # [ 181.269763] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)6255client # [ 181.276290] nix[1555]: copying 1 paths to 'lb:50051'...6256worker1 # [ 181.353526] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166257client # [ 181.278660] nix[1555]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6258worker1 # [ 181.370173] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6259worker2 # [ 181.342951] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5)6260worker1 # [ 181.526657] nix-daemon[854]: accepted connection from pid 1950, user nix-grpc-daemon (trusted)6261worker1 # [ 181.546756] nix-daemon[854]: reaped child process 1953, status = succeeded6262lb # [ 181.475135] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received uploads request method=POST path=/api/pending_closures6263worker1 # [ 181.561612] nix-grpc-daemon[1777]: time=2026-09-23T05:35:01.164Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6264worker1 # [ 181.563173] nix-grpc-daemon[1777]: time=2026-09-23T05:35:01.165Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6265lb # [ 181.506501] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6266worker1 # [ 181.592863] nix-grpc-daemon[1777]: time=2026-09-23T05:35:01.196Z level=INFO msg="Uploading 1 narinfos"6267lb # [ 181.521985] niks3-server[1137]: 2026/09/23 05:35:01 INFO Signed narinfos id=62 count=16268lb # [ 181.529332] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6269worker2 # [ 181.539280] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 3/5)6270lb # [ 181.539123] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6271worker1 # [ 181.624541] nix-grpc-daemon[1777]: time=2026-09-23T05:35:01.228Z level=INFO msg="Upload complete. (238ms)"6272lb # [ 181.548298] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6273lb # [ 181.551731] niks3-server[1137]: 2026/09/23 05:35:01 INFO Completed upload id=626274lb # [ 181.555307] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6275worker1 # [ 181.637854] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=4166276worker1 # [ 181.652001] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166277worker1 # [ 181.661438] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6278worker1 # [ 181.664367] nix-daemon[854]: reaped child process 1948, status = succeeded6279client # [ 181.598182] nix[1555]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6280worker1 # [ 181.678118] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6281worker2 # [ 181.886107] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 4/5)6282worker2 # [ 181.899577] nix-grpc-daemon[1775]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6283worker2 # [ 181.906617] nix-grpc-daemon[1775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6284worker2 # [ 181.924208] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331006285client # [ 181.931844] nix[1555]: copying 1 paths to 'lb:50051'...6286worker2 # [ 181.938959] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331006287client # [ 181.934350] nix[1555]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6288worker2 # [ 181.953317] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6289worker2 # [ 181.980848] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=info event=niks3_push_started pid=18366290worker2 # [ 182.128422] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.744Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6291lb: must succeed: curl -sf localhost:9901/clusters6292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6293lb: must succeed: curl -sf localhost:9901/clusters6294worker2 # [ 182.238367] nix-daemon[839]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6295worker2 # [ 182.254316] nix-daemon[839]: reaped child process 1852, status = succeeded6296lb # [ 182.258608] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received uploads request method=POST path=/api/pending_closures6297lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6298worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6299worker2 # [ 182.270520] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.885Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6300worker2 # [ 182.271936] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.885Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6301lb # [ 182.284414] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6302worker2 # [ 182.300557] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.916Z level=INFO msg="Uploading 1 narinfos"6303lb # [ 182.293288] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6304lb # [ 182.300838] niks3-server[1137]: 2026/09/23 05:35:01 INFO Signed narinfos id=63 count=16305lb # [ 182.307350] niks3-server[1137]: 2026/09/23 05:35:01 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6306lb # [ 182.315560] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6307worker2 # [ 182.326320] nix-grpc-daemon[1836]: time=2026-09-23T05:35:01.942Z level=INFO msg="Upload complete. (194ms)"6308lb # [ 182.320269] niks3-server[1137]: 2026/09/23 05:35:01 INFO Completed upload id=636309worker2 # [ 182.332282] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=4166310lb # [ 182.325136] niks3-server[1137]: 2026/09/23 05:35:01 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6311worker2 # [ 182.342004] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331006312worker2 # [ 182.347170] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6313worker2 # [ 182.350622] nix-daemon[839]: reaped child process 1834, status = succeeded6314worker2 # [ 182.360189] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6315client # [ 182.352674] nix[1555]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6316worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)6317worker1 # [ 183.708838] nix-daemon[854]: reaped child process 1957, status = succeeded6318worker1 # [ 183.795302] nix-daemon[854]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6319worker1 # [ 183.814522] nix-daemon[854]: reaped child process 1988, status = succeeded6320lb # [ 183.744089] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures6321worker1 # [ 183.832214] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.433Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6322worker1 # [ 183.833657] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.433Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6323lb # [ 183.786732] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6324worker1 # [ 183.871977] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.475Z level=INFO msg="Uploading 1 narinfos"6325worker2 # [ 183.815080] nix-daemon[839]: reaped child process 1856, status = succeeded6326lb # [ 183.801782] niks3-server[1137]: 2026/09/23 05:35:03 INFO Signed narinfos id=64 count=16327lb # [ 183.810881] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6328lb # [ 183.821141] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6329worker1 # [ 183.904636] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.508Z level=INFO msg="Upload complete. (216ms)"6330lb # [ 183.829956] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6331lb # [ 183.836940] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6332lb # [ 183.841242] niks3-server[1137]: 2026/09/23 05:35:03 INFO Completed upload id=646333worker1 # [ 183.916824] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=16334lb # [ 183.843719] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6335worker1 # [ 183.930499] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=16336worker1 # [ 183.934528] nix-daemon[854]: reaped child process 1955, status = succeeded6337worker1 # [ 183.958460] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166338worker1 # [ 183.967606] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166339client # [ 183.905268] nix[1555]: copying 1 paths to 'lb:50051'...6340worker1 # [ 183.981672] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166341client # [ 183.905537] nix[1555]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6342worker2 # [ 183.920130] nix-daemon[839]: accepted connection from pid 1881, user nix-grpc-daemon (trusted)6343worker1 # [ 183.997899] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6344worker2 # [ 183.938333] nix-daemon[839]: reaped child process 1884, status = succeeded6345lb # [ 183.938882] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures6346worker2 # [ 183.956342] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.570Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6347worker2 # [ 183.957896] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.570Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6348lb # [ 183.982639] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6349worker2 # [ 184.001369] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.617Z level=INFO msg="Uploading 1 narinfos"6350lb # [ 183.994708] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6351lb # [ 184.003358] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6352lb # [ 184.012580] niks3-server[1137]: 2026/09/23 05:35:03 INFO Signed narinfos id=65 count=16353worker2 # [ 184.019426] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.635Z level=INFO msg="Upload complete. (250ms)"6354lb # [ 184.015459] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6355worker1 # [ 184.096368] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=16356lb # [ 184.020804] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6357lb # [ 184.024287] niks3-server[1137]: 2026/09/23 05:35:03 INFO Completed upload id=656358worker2 # [ 184.029038] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=16359lb # [ 184.027038] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6360worker2 # [ 184.048081] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331006361worker2 # [ 184.052935] nix-daemon[839]: reaped child process 1854, status = succeeded6362worker2 # [ 184.059245] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331006363client # [ 184.060618] nix[1555]: copying 1 paths to 'lb:50051'...6364client # [ 184.065439] nix[1555]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6365worker2 # [ 184.073609] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331006366worker2 # [ 184.083513] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6367worker1 # [ 184.170803] nix-daemon[854]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6368worker1 # [ 184.191007] nix-daemon[854]: reaped child process 1995, status = succeeded6369lb # [ 184.112835] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures6370worker1 # [ 184.198391] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.802Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6371worker1 # [ 184.200199] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.802Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6372lb # [ 184.142988] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6373worker1 # [ 184.229530] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.833Z level=INFO msg="Uploading 1 narinfos"6374lb # [ 184.155217] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6375lb # [ 184.163810] niks3-server[1137]: 2026/09/23 05:35:03 INFO Signed narinfos id=66 count=16376lb # [ 184.167494] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6377worker1 # [ 184.253134] nix-grpc-daemon[1777]: time=2026-09-23T05:35:03.857Z level=INFO msg="Upload complete. (240ms)"6378lb # [ 184.178031] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6379lb # [ 184.182209] niks3-server[1137]: 2026/09/23 05:35:03 INFO Completed upload id=666380lb # [ 184.184779] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6381worker1 # [ 184.260102] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=4166382worker1 # [ 184.274153] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166383worker1 # [ 184.279712] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6384worker1 # [ 184.284853] nix-daemon[854]: reaped child process 1990, status = succeeded6385worker2 # [ 184.219969] nix-daemon[839]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6386worker1 # [ 184.295471] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6387client # [ 184.217228] nix[1555]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6388worker2 # [ 184.239872] nix-daemon[839]: reaped child process 1891, status = succeeded6389lb # [ 184.238649] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures6390worker2 # [ 184.253902] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6391worker2 # [ 184.255376] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.870Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6392lb # [ 184.267217] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6393worker2 # [ 184.279813] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.895Z level=INFO msg="Uploading 1 narinfos"6394lb # [ 184.280882] niks3-server[1137]: 2026/09/23 05:35:03 INFO Signed narinfos id=67 count=16395lb # [ 184.283261] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6396worker2 # [ 184.303019] nix-grpc-daemon[1836]: time=2026-09-23T05:35:03.919Z level=INFO msg="Upload complete. (207ms)"6397lb # [ 184.296315] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6398lb # [ 184.301323] niks3-server[1137]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6399lb # [ 184.305344] niks3-server[1137]: 2026/09/23 05:35:03 INFO Completed upload id=676400worker2 # [ 184.312712] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=4166401lb # [ 184.309035] niks3-server[1137]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6402worker2 # [ 184.321873] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331006403worker2 # [ 184.327708] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6404worker2 # [ 184.331781] nix-daemon[839]: reaped child process 1886, status = succeeded6405worker2 # [ 184.341793] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6406client # [ 184.333729] nix[1555]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6407worker1 # [ 186.124988] nix-daemon[854]: reaped child process 1999, status = succeeded6408worker1 # [ 186.218908] nix-daemon[854]: accepted connection from pid 2024, user nix-grpc-daemon (trusted)6409worker1 # [ 186.238658] nix-daemon[854]: reaped child process 2027, status = succeeded6410lb # [ 186.166615] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received uploads request method=POST path=/api/pending_closures6411worker2 # [ 186.180982] nix-daemon[839]: reaped child process 1895, status = succeeded6412worker1 # [ 186.256386] nix-grpc-daemon[1777]: time=2026-09-23T05:35:05.858Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6413worker1 # [ 186.257871] nix-grpc-daemon[1777]: time=2026-09-23T05:35:05.858Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6414lb # [ 186.211976] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6415worker1 # [ 186.298272] nix-grpc-daemon[1777]: time=2026-09-23T05:35:05.902Z level=INFO msg="Uploading 1 narinfos"6416lb # [ 186.225845] niks3-server[1137]: 2026/09/23 05:35:05 INFO Signed narinfos id=68 count=16417lb # [ 186.233860] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6418lb # [ 186.242150] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6419lb # [ 186.252401] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6420worker1 # [ 186.327977] nix-grpc-daemon[1777]: time=2026-09-23T05:35:05.931Z level=INFO msg="Upload complete. (219ms)"6421lb # [ 186.254972] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6422lb # [ 186.259702] niks3-server[1137]: 2026/09/23 05:35:05 INFO Completed upload id=686423lb # [ 186.261703] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6424worker1 # [ 186.340782] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=16425worker1 # [ 186.352170] nix-daemon[854]: reaped child process 1997, status = succeeded6426worker2 # [ 186.287007] nix-daemon[839]: accepted connection from pid 1920, user nix-grpc-daemon (trusted)6427worker2 # [ 186.304352] nix-daemon[839]: reaped child process 1923, status = succeeded6428lb # [ 186.303330] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received uploads request method=POST path=/api/pending_closures6429worker2 # [ 186.319282] nix-grpc-daemon[1836]: time=2026-09-23T05:35:05.934Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6430worker2 # [ 186.320918] nix-grpc-daemon[1836]: time=2026-09-23T05:35:05.934Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6431lb # [ 186.336745] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6432worker2 # [ 186.349890] nix-grpc-daemon[1836]: time=2026-09-23T05:35:05.966Z level=INFO msg="Uploading 1 narinfos"6433lb # [ 186.350162] niks3-server[1137]: 2026/09/23 05:35:05 INFO Signed narinfos id=69 count=16434lb # [ 186.355834] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6435worker2 # [ 186.376476] nix-grpc-daemon[1836]: time=2026-09-23T05:35:05.992Z level=INFO msg="Upload complete. (223ms)"6436lb # [ 186.369586] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6437lb # [ 186.374070] niks3-server[1137]: 2026/09/23 05:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6438lb # [ 186.378502] niks3-server[1137]: 2026/09/23 05:35:05 INFO Completed upload id=696439lb # [ 186.383851] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6440lb # [ 186.388997] niks3-server[1137]: 2026/09/23 05:35:05 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6441worker2 # [ 186.385158] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=16442worker2 # [ 186.399860] nix-daemon[839]: reaped child process 1893, status = succeeded6443client # [ 186.418029] systemd[1]: burst.service: Deactivated successfully.6444client # [ 186.423058] systemd[1]: burst.service: Consumed 224ms CPU time over 5.474s wall clock time, 51.7M memory peak, 14.2K incoming IP traffic, 18.9K outgoing IP traffic.6445client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6446client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6447lb: must succeed: curl -sf localhost:9901/clusters6448lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6449lb: must succeed: curl -sf localhost:9901/clusters6450lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6451worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6452worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6453(finished: subtest: queue gauges show a burst while every build blocks, in 6.87 seconds)6454subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6455client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16456worker1 # [ 187.888856] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478166457worker1 # [ 187.895439] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166458worker1 # [ 187.937124] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06459worker1 # [ 187.944137] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16460worker1 # [ 187.950028] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06461worker1 # [ 187.955287] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06462worker1 # [ 187.985386] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166463worker1 # [ 188.000526] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166464worker1 # [ 188.014972] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166465worker1 # [ 188.029622] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6466worker1 # [ 188.182802] nix-daemon[854]: accepted connection from pid 2040, user nix-grpc-daemon (trusted)6467worker1 # [ 188.201585] nix-daemon[854]: reaped child process 2043, status = succeeded6468lb # [ 188.129774] niks3-server[1137]: 2026/09/23 05:35:07 INFO Received uploads request method=POST path=/api/pending_closures6469worker1 # [ 188.218577] nix-grpc-daemon[1777]: time=2026-09-23T05:35:07.821Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6470worker1 # [ 188.220353] nix-grpc-daemon[1777]: time=2026-09-23T05:35:07.821Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6471lb # [ 188.162622] niks3-server[1137]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6472worker1 # [ 188.251418] nix-grpc-daemon[1777]: time=2026-09-23T05:35:07.855Z level=INFO msg="Uploading 1 narinfos"6473lb # [ 188.177742] niks3-server[1137]: 2026/09/23 05:35:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6474lb # [ 188.186736] niks3-server[1137]: 2026/09/23 05:35:07 INFO Signed narinfos id=70 count=16475worker1 # [ 188.273557] nix-grpc-daemon[1777]: time=2026-09-23T05:35:07.877Z level=INFO msg="Upload complete. (228ms)"6476lb # [ 188.196603] niks3-server[1137]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6477worker1 # [ 188.282738] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=5126478lb # [ 188.205252] niks3-server[1137]: 2026/09/23 05:35:07 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6479lb # [ 188.211894] niks3-server[1137]: 2026/09/23 05:35:07 INFO Completed upload id=706480lb # [ 188.215445] niks3-server[1137]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6481worker1 # [ 188.296882] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166482worker1 # [ 188.305998] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6483worker1 # [ 188.311469] nix-daemon[854]: reaped child process 2038, status = succeeded6484worker1 # [ 188.322133] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6485worker1 # [ 189.631966] nix-daemon[854]: reaped child process 2047, status = succeeded6486worker1 # [ 189.705626] nix-daemon[854]: accepted connection from pid 2063, user nix-grpc-daemon (trusted)6487lb # [ 189.627431] envoy[904]: conn peer=[2001:db8:1::1]:47760 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9688 tx=5198 ms=52526488worker1 # [ 189.723413] nix-daemon[854]: reaped child process 2066, status = succeeded6489lb # [ 189.652226] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received uploads request method=POST path=/api/pending_closures6490worker1 # [ 189.739629] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.342Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6491worker1 # [ 189.741443] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.342Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6492worker1 # [ 189.780915] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.384Z level=INFO msg="Uploading 1 narinfos"6493lb # [ 189.693362] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6494lb # [ 189.709788] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6495lb # [ 189.719734] niks3-server[1137]: 2026/09/23 05:35:09 INFO Signed narinfos id=71 count=16496lb # [ 189.725287] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6497worker1 # [ 189.813400] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.417Z level=INFO msg="Upload complete. (278ms)"6498lb # [ 189.731871] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6499lb # [ 189.740842] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6500lb # [ 189.745962] niks3-server[1137]: 2026/09/23 05:35:09 INFO Completed upload id=716501lb # [ 189.749045] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6502worker1 # [ 189.833057] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=16503worker1 # [ 189.848782] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16504worker1 # [ 189.855554] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166505worker1 # [ 189.861137] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06506worker1 # [ 189.866249] nix-daemon[854]: reaped child process 2045, status = succeeded6507worker1 # [ 189.870290] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166508worker1 # [ 189.879527] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166509worker1 # [ 189.893105] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6510worker1 # [ 190.022697] nix-daemon[854]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)6511worker1 # [ 190.039597] nix-daemon[854]: reaped child process 2075, status = succeeded6512lb # [ 189.968781] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received uploads request method=POST path=/api/pending_closures6513worker1 # [ 190.054202] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.657Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6514worker1 # [ 190.055689] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.657Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6515lb # [ 189.998192] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6516worker1 # [ 190.088255] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.692Z level=INFO msg="Uploading 1 narinfos"6517lb # [ 190.013514] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6518lb # [ 190.024676] niks3-server[1137]: 2026/09/23 05:35:09 INFO Signed narinfos id=72 count=16519lb # [ 190.034846] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6520worker1 # [ 190.117030] nix-grpc-daemon[1777]: time=2026-09-23T05:35:09.721Z level=INFO msg="Upload complete. (215ms)"6521lb # [ 190.045998] niks3-server[1137]: 2026/09/23 05:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6522worker1 # [ 190.124137] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=5126523lb # [ 190.051618] niks3-server[1137]: 2026/09/23 05:35:09 INFO Completed upload id=726524lb # [ 190.054622] niks3-server[1137]: 2026/09/23 05:35:09 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6525worker1 # [ 190.139505] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166526worker1 # [ 190.145949] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6527worker1 # [ 190.150452] nix-daemon[854]: reaped child process 2070, status = succeeded6528worker1 # [ 190.161173] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6529worker1 # [ 191.636677] nix-daemon[854]: reaped child process 2079, status = succeeded6530worker1 # [ 191.734970] nix-daemon[854]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)6531worker1 # [ 191.752999] nix-daemon[854]: reaped child process 2098, status = succeeded6532lb # [ 191.681034] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received uploads request method=POST path=/api/pending_closures6533worker1 # [ 191.768402] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.371Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6534worker1 # [ 191.772727] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.371Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6535lb # [ 191.720816] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6536worker1 # [ 191.810191] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.414Z level=INFO msg="Uploading 1 narinfos"6537lb # [ 191.736319] niks3-server[1137]: 2026/09/23 05:35:11 INFO Signed narinfos id=73 count=16538lb # [ 191.739108] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6539lb # [ 191.751854] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6540lb # [ 191.763382] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6541worker1 # [ 191.840287] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.444Z level=INFO msg="Upload complete. (267ms)"6542lb # [ 191.768264] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6543lb # [ 191.772892] niks3-server[1137]: 2026/09/23 05:35:11 INFO Completed upload id=736544lb # [ 191.775893] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6545worker1 # [ 191.851223] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=16546worker1 # [ 191.876297] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16547worker1 # [ 191.881485] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166548worker1 # [ 191.886913] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06549worker1 # [ 191.892112] nix-daemon[854]: reaped child process 2077, status = succeeded6550worker1 # [ 191.894009] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166551worker1 # [ 191.902456] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166552worker1 # [ 191.913197] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6553worker1 # [ 192.050670] nix-daemon[854]: accepted connection from pid 2104, user nix-grpc-daemon (trusted)6554worker1 # [ 192.067311] nix-daemon[854]: reaped child process 2107, status = succeeded6555lb # [ 191.996999] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received uploads request method=POST path=/api/pending_closures6556worker1 # [ 192.082322] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.686Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6557worker1 # [ 192.083777] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.686Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6558lb # [ 192.025255] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6559worker1 # [ 192.108542] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.712Z level=INFO msg="Uploading 1 narinfos"6560lb # [ 192.039369] niks3-server[1137]: 2026/09/23 05:35:11 INFO Signed narinfos id=74 count=16561lb # [ 192.047112] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6562lb # [ 192.060870] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6563worker1 # [ 192.143803] nix-grpc-daemon[1777]: time=2026-09-23T05:35:11.747Z level=INFO msg="Upload complete. (219ms)"6564lb # [ 192.067778] niks3-server[1137]: 2026/09/23 05:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6565lb # [ 192.078169] niks3-server[1137]: 2026/09/23 05:35:11 INFO Completed upload id=746566lb # [ 192.081000] niks3-server[1137]: 2026/09/23 05:35:11 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6567worker1 # [ 192.152273] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=5126568worker1 # [ 192.166799] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166569worker1 # [ 192.175463] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6570worker1 # [ 192.178148] nix-daemon[854]: reaped child process 2102, status = succeeded6571worker1 # [ 192.190465] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6572worker1 # [ 193.418935] nix-daemon[854]: reaped child process 2111, status = succeeded6573worker1 # [ 193.522794] nix-daemon[854]: accepted connection from pid 2127, user nix-grpc-daemon (trusted)6574worker1 # [ 193.540460] nix-daemon[854]: reaped child process 2130, status = succeeded6575lb # [ 193.471037] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received uploads request method=POST path=/api/pending_closures6576worker1 # [ 193.559257] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.162Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6577worker1 # [ 193.560980] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.162Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6578lb # [ 193.516325] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6579worker1 # [ 193.600108] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.203Z level=INFO msg="Uploading 1 narinfos"6580lb # [ 193.528787] niks3-server[1137]: 2026/09/23 05:35:13 INFO Signed narinfos id=75 count=16581lb # [ 193.531344] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6582lb # [ 193.542881] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6583lb # [ 193.547520] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6584lb # [ 193.557081] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6585worker1 # [ 193.633281] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.237Z level=INFO msg="Upload complete. (232ms)"6586lb # [ 193.560042] niks3-server[1137]: 2026/09/23 05:35:13 INFO Completed upload id=756587lb # [ 193.563558] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6588worker1 # [ 193.649625] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=16589worker1 # [ 193.668339] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16590worker1 # [ 193.673374] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166591worker1 # [ 193.678437] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06592worker1 # [ 193.683131] nix-daemon[854]: reaped child process 2109, status = succeeded6593worker1 # [ 193.686145] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166594worker1 # [ 193.698013] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.302Z level=INFO msg="All 1 paths already cached"6595worker1 # [ 193.705664] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166596worker1 # [ 193.718767] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6597worker1 # [ 193.850815] nix-daemon[854]: accepted connection from pid 2136, user nix-grpc-daemon (trusted)6598worker1 # [ 193.868457] nix-daemon[854]: reaped child process 2139, status = succeeded6599lb # [ 193.798804] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received uploads request method=POST path=/api/pending_closures6600worker1 # [ 193.883931] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.487Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6601worker1 # [ 193.886161] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.489Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6602lb # [ 193.833974] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6603worker1 # [ 193.917189] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.521Z level=INFO msg="Uploading 1 narinfos"6604lb # [ 193.848428] niks3-server[1137]: 2026/09/23 05:35:13 INFO Signed narinfos id=76 count=16605lb # [ 193.851452] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6606worker1 # [ 193.942152] nix-grpc-daemon[1777]: time=2026-09-23T05:35:13.546Z level=INFO msg="Upload complete. (214ms)"6607lb # [ 193.866796] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6608lb # [ 193.873304] niks3-server[1137]: 2026/09/23 05:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6609lb # [ 193.877935] niks3-server[1137]: 2026/09/23 05:35:13 INFO Completed upload id=766610worker1 # [ 193.952298] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=6246611lb # [ 193.880958] niks3-server[1137]: 2026/09/23 05:35:13 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6612worker1 # [ 193.965612] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166613worker1 # [ 193.971693] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6614worker1 # [ 193.976668] nix-daemon[854]: reaped child process 2134, status = succeeded6615worker1 # [ 193.988462] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6616worker1 # [ 195.092419] nix-daemon[854]: reaped child process 2143, status = succeeded6617worker1 # [ 195.166315] nix-daemon[854]: accepted connection from pid 2159, user nix-grpc-daemon (trusted)6618worker1 # [ 195.184962] nix-daemon[854]: reaped child process 2162, status = succeeded6619lb # [ 195.114805] niks3-server[1137]: 2026/09/23 05:35:14 INFO Received uploads request method=POST path=/api/pending_closures6620worker1 # [ 195.203922] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.805Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6621worker1 # [ 195.206010] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.805Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6622lb # [ 195.148983] niks3-server[1137]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6623lb # [ 195.161875] niks3-server[1137]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6624worker1 # [ 195.253714] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.857Z level=INFO msg="Uploading 1 narinfos"6625lb # [ 195.176684] niks3-server[1137]: 2026/09/23 05:35:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6626lb # [ 195.189889] niks3-server[1137]: 2026/09/23 05:35:14 INFO Signed narinfos id=77 count=16627lb # [ 195.197152] niks3-server[1137]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6628worker1 # [ 195.285561] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.889Z level=INFO msg="Upload complete. (287ms)"6629lb # [ 195.210298] niks3-server[1137]: 2026/09/23 05:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6630lb # [ 195.218125] niks3-server[1137]: 2026/09/23 05:35:14 INFO Completed upload id=776631lb # [ 195.221614] niks3-server[1137]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6632worker1 # [ 195.303393] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=16633worker1 # [ 195.324918] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16634worker1 # [ 195.335884] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166635worker1 # [ 195.342601] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06636worker1 # [ 195.348915] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166637worker1 # [ 195.367011] nix-grpc-daemon[1777]: time=2026-09-23T05:35:14.951Z level=INFO msg="All 1 paths already cached"6638worker1 # [ 195.370756] nix-daemon[854]: reaped child process 2141, status = succeeded6639worker1 # [ 195.373385] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6640worker1 # [ 195.375539] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166641worker1 # [ 195.505568] nix-daemon[854]: accepted connection from pid 2168, user nix-grpc-daemon (trusted)6642worker1 # [ 195.521410] nix-daemon[854]: reaped child process 2171, status = succeeded6643lb # [ 195.449799] niks3-server[1137]: 2026/09/23 05:35:15 INFO Received uploads request method=POST path=/api/pending_closures6644worker1 # [ 195.535230] nix-grpc-daemon[1777]: time=2026-09-23T05:35:15.138Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6645worker1 # [ 195.536803] nix-grpc-daemon[1777]: time=2026-09-23T05:35:15.138Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6646lb # [ 195.476786] niks3-server[1137]: 2026/09/23 05:35:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6647worker1 # [ 195.561335] nix-grpc-daemon[1777]: time=2026-09-23T05:35:15.165Z level=INFO msg="Uploading 1 narinfos"6648lb # [ 195.492682] niks3-server[1137]: 2026/09/23 05:35:15 INFO Signed narinfos id=78 count=16649lb # [ 195.495892] niks3-server[1137]: 2026/09/23 05:35:15 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6650worker1 # [ 195.586825] nix-grpc-daemon[1777]: time=2026-09-23T05:35:15.190Z level=INFO msg="Upload complete. (206ms)"6651lb # [ 195.509907] niks3-server[1137]: 2026/09/23 05:35:15 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6652lb # [ 195.517046] niks3-server[1137]: 2026/09/23 05:35:15 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6653worker1 # [ 195.594715] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=6246654lb # [ 195.521547] niks3-server[1137]: 2026/09/23 05:35:15 INFO Completed upload id=786655lb # [ 195.523594] niks3-server[1137]: 2026/09/23 05:35:15 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6656worker1 # [ 195.604734] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166657worker1 # [ 195.613978] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6658worker1 # [ 195.618230] nix-daemon[854]: reaped child process 2166, status = succeeded6659worker1 # [ 195.627916] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6660worker1 # [ 196.496775] nix-daemon[854]: reaped child process 2175, status = succeeded6661worker1 # [ 196.597819] nix-daemon[854]: accepted connection from pid 2191, user nix-grpc-daemon (trusted)6662worker1 # [ 196.616361] nix-daemon[854]: reaped child process 2194, status = succeeded6663lb # [ 196.543889] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received uploads request method=POST path=/api/pending_closures6664worker1 # [ 196.630880] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.233Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6665worker1 # [ 196.632616] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.233Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6666lb # [ 196.584984] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6667worker1 # [ 196.673134] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.276Z level=INFO msg="Uploading 1 narinfos"6668lb # [ 196.599517] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6669lb # [ 196.610623] niks3-server[1137]: 2026/09/23 05:35:16 INFO Signed narinfos id=79 count=16670lb # [ 196.615335] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6671lb # [ 196.622928] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6672worker1 # [ 196.705687] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.309Z level=INFO msg="Upload complete. (267ms)"6673lb # [ 196.629308] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6674lb # [ 196.633004] niks3-server[1137]: 2026/09/23 05:35:16 INFO Completed upload id=796675lb # [ 196.637551] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6676worker1 # [ 196.722934] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=16677worker1 # [ 196.739696] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16678worker1 # [ 196.750699] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06679worker1 # [ 196.756179] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166680worker1 # [ 196.762202] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166681worker1 # [ 196.767861] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.370Z level=INFO msg="All 1 paths already cached"6682worker1 # [ 196.783774] nix-daemon[854]: reaped child process 2173, status = succeeded6683worker1 # [ 196.785893] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6684worker1 # [ 196.790670] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166685worker1 # [ 196.916941] nix-daemon[854]: accepted connection from pid 2200, user nix-grpc-daemon (trusted)6686worker1 # [ 196.934520] nix-daemon[854]: reaped child process 2203, status = succeeded6687lb # [ 196.860338] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received uploads request method=POST path=/api/pending_closures6688worker1 # [ 196.945707] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.549Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6689worker1 # [ 196.947150] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.549Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6690lb # [ 196.892237] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6691worker1 # [ 196.978392] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.582Z level=INFO msg="Uploading 1 narinfos"6692lb # [ 196.905394] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6693lb # [ 196.915332] niks3-server[1137]: 2026/09/23 05:35:16 INFO Signed narinfos id=80 count=16694lb # [ 196.919242] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6695worker1 # [ 197.005834] nix-grpc-daemon[1777]: time=2026-09-23T05:35:16.609Z level=INFO msg="Upload complete. (211ms)"6696lb # [ 196.929845] niks3-server[1137]: 2026/09/23 05:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6697lb # [ 196.934414] niks3-server[1137]: 2026/09/23 05:35:16 INFO Completed upload id=806698worker1 # [ 197.012325] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=6246699lb # [ 196.937675] niks3-server[1137]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6700worker1 # [ 197.024751] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166701worker1 # [ 197.030232] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6702worker1 # [ 197.035365] nix-daemon[854]: reaped child process 2198, status = succeeded6703worker1 # [ 197.045075] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6704worker1 # [ 197.942908] nix-daemon[854]: reaped child process 2207, status = succeeded6705worker1 # [ 198.012888] nix-daemon[854]: accepted connection from pid 2223, user nix-grpc-daemon (trusted)6706worker1 # [ 198.032970] nix-daemon[854]: reaped child process 2226, status = succeeded6707lb # [ 197.955158] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received uploads request method=POST path=/api/pending_closures6708worker1 # [ 198.042808] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.644Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6709worker1 # [ 198.044370] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.644Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6710lb # [ 197.995992] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6711lb # [ 198.005571] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6712worker1 # [ 198.093047] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.696Z level=INFO msg="Uploading 1 narinfos"6713lb # [ 198.020444] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6714lb # [ 198.029096] niks3-server[1137]: 2026/09/23 05:35:17 INFO Signed narinfos id=81 count=16715lb # [ 198.031838] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6716worker1 # [ 198.122112] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.726Z level=INFO msg="Upload complete. (266ms)"6717lb # [ 198.046976] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete6718lb # [ 198.050783] niks3-server[1137]: 2026/09/23 05:35:17 INFO Completed upload id=816719lb # [ 198.054152] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6720worker1 # [ 198.136803] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=16721worker1 # [ 198.152285] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16722worker1 # [ 198.158317] nix-daemon[854]: reaped child process 2205, status = succeeded6723worker1 # [ 198.179810] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166724worker1 # [ 198.188121] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06725worker1 # [ 198.199393] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166726worker1 # [ 198.213216] nix-grpc-daemon[1777]: time=2026-09-23T05:35:17.817Z level=INFO msg="All 2 paths already cached"6727worker1 # [ 198.223192] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166728worker1 # [ 198.239423] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6729worker1 # [ 198.377933] nix-daemon[854]: accepted connection from pid 2232, user nix-grpc-daemon (trusted)6730worker1 # [ 198.396944] nix-daemon[854]: reaped child process 2235, status = succeeded6731lb # [ 198.325751] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received uploads request method=POST path=/api/pending_closures6732worker1 # [ 198.411216] nix-grpc-daemon[1777]: time=2026-09-23T05:35:18.014Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6733worker1 # [ 198.412849] nix-grpc-daemon[1777]: time=2026-09-23T05:35:18.014Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6734worker1 # [ 198.440767] nix-grpc-daemon[1777]: time=2026-09-23T05:35:18.044Z level=INFO msg="Uploading 1 narinfos"6735lb # [ 198.357929] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign6736lb # [ 198.373701] niks3-server[1137]: 2026/09/23 05:35:17 INFO Signed narinfos id=82 count=16737lb # [ 198.380771] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6738lb # [ 198.387252] niks3-server[1137]: 2026/09/23 05:35:17 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6739worker1 # [ 198.474528] nix-grpc-daemon[1777]: time=2026-09-23T05:35:18.078Z level=INFO msg="Upload complete. (224ms)"6740lb # [ 198.396732] niks3-server[1137]: 2026/09/23 05:35:17 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete6741lb # [ 198.406335] niks3-server[1137]: 2026/09/23 05:35:18 INFO Completed upload id=826742worker1 # [ 198.485095] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=7526743lb # [ 198.410193] niks3-server[1137]: 2026/09/23 05:35:18 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6744worker1 # [ 198.499970] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166745worker1 # [ 198.508163] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6746worker1 # [ 198.511074] nix-daemon[854]: reaped child process 2230, status = succeeded6747worker1 # [ 198.521058] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6748worker1 # [ 199.622620] nix-daemon[854]: reaped child process 2239, status = succeeded6749worker1 # [ 199.700993] nix-daemon[854]: accepted connection from pid 2255, user nix-grpc-daemon (trusted)6750worker1 # [ 199.722944] nix-daemon[854]: reaped child process 2258, status = succeeded6751lb # [ 199.645253] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received uploads request method=POST path=/api/pending_closures6752worker1 # [ 199.734592] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.337Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6753worker1 # [ 199.736081] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.337Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6754lb # [ 199.691050] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6755lb # [ 199.702759] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign6756worker1 # [ 199.784170] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.387Z level=INFO msg="Uploading 1 narinfos"6757lb # [ 199.713848] niks3-server[1137]: 2026/09/23 05:35:19 INFO Signed narinfos id=83 count=16758lb # [ 199.718242] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6759lb # [ 199.727523] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6760lb # [ 199.732884] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete6761worker1 # [ 199.809476] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.413Z level=INFO msg="Upload complete. (277ms)"6762lb # [ 199.735987] niks3-server[1137]: 2026/09/23 05:35:19 INFO Completed upload id=836763lb # [ 199.739291] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6764worker1 # [ 199.821880] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=16765worker1 # [ 199.838984] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16766worker1 # [ 199.843959] nix-daemon[854]: reaped child process 2237, status = succeeded6767worker1 # [ 199.865227] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166768worker1 # [ 199.880615] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166769worker1 # [ 199.894329] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.498Z level=INFO msg="All 2 paths already cached"6770worker1 # [ 199.904147] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166771worker1 # [ 199.919957] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6772worker1 # [ 200.067134] nix-daemon[854]: accepted connection from pid 2262, user nix-grpc-daemon (trusted)6773worker1 # [ 200.085431] nix-daemon[854]: reaped child process 2265, status = succeeded6774lb # [ 200.011325] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received uploads request method=POST path=/api/pending_closures6775worker1 # [ 200.096776] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.700Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6776worker1 # [ 200.098305] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.700Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6777lb # [ 200.040531] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign6778worker1 # [ 200.125422] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.729Z level=INFO msg="Uploading 1 narinfos"6779lb # [ 200.055837] niks3-server[1137]: 2026/09/23 05:35:19 INFO Signed narinfos id=84 count=16780lb # [ 200.062627] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6781lb # [ 200.073593] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6782worker1 # [ 200.155551] nix-grpc-daemon[1777]: time=2026-09-23T05:35:19.759Z level=INFO msg="Upload complete. (220ms)"6783lb # [ 200.079245] niks3-server[1137]: 2026/09/23 05:35:19 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete6784lb # [ 200.088697] niks3-server[1137]: 2026/09/23 05:35:19 INFO Completed upload id=846785worker1 # [ 200.163799] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=7526786lb # [ 200.090923] niks3-server[1137]: 2026/09/23 05:35:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6787worker1 # [ 200.178190] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166788worker1 # [ 200.187717] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6789worker1 # [ 200.190897] nix-daemon[854]: reaped child process 2260, status = succeeded6790worker1 # [ 200.202147] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6791worker1 # [ 201.028664] nix-daemon[854]: reaped child process 2269, status = succeeded6792worker1 # [ 201.130512] nix-daemon[854]: accepted connection from pid 2285, user nix-grpc-daemon (trusted)6793worker1 # [ 201.145914] nix-daemon[854]: reaped child process 2288, status = succeeded6794lb # [ 201.076941] niks3-server[1137]: 2026/09/23 05:35:20 INFO Received uploads request method=POST path=/api/pending_closures6795worker1 # [ 201.164379] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.766Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6796worker1 # [ 201.165856] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.766Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6797lb # [ 201.121046] niks3-server[1137]: 2026/09/23 05:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign6798worker1 # [ 201.206925] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.810Z level=INFO msg="Uploading 1 narinfos"6799lb # [ 201.134642] niks3-server[1137]: 2026/09/23 05:35:20 INFO Signed narinfos id=85 count=16800lb # [ 201.142291] niks3-server[1137]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6801lb # [ 201.151829] niks3-server[1137]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6802worker1 # [ 201.238677] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.842Z level=INFO msg="Upload complete. (225ms)"6803lb # [ 201.162355] niks3-server[1137]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6804lb # [ 201.167165] niks3-server[1137]: 2026/09/23 05:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete6805lb # [ 201.173447] niks3-server[1137]: 2026/09/23 05:35:20 INFO Completed upload id=856806lb # [ 201.178555] niks3-server[1137]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6807worker1 # [ 201.252243] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=16808worker1 # [ 201.267270] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16809worker1 # [ 201.272305] nix-daemon[854]: reaped child process 2267, status = succeeded6810worker1 # [ 201.297008] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166811worker1 # [ 201.312424] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478166812worker1 # [ 201.326869] nix-grpc-daemon[1777]: time=2026-09-23T05:35:20.930Z level=INFO msg="All 2 paths already cached"6813worker1 # [ 201.336693] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478166814worker1 # [ 201.352943] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6815worker1 # [ 201.494469] nix-daemon[854]: accepted connection from pid 2292, user nix-grpc-daemon (trusted)6816worker1 # [ 201.513489] nix-daemon[854]: reaped child process 2295, status = succeeded6817lb # [ 201.434794] niks3-server[1137]: 2026/09/23 05:35:21 INFO Received uploads request method=POST path=/api/pending_closures6818worker1 # [ 201.523810] nix-grpc-daemon[1777]: time=2026-09-23T05:35:21.127Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6819worker1 # [ 201.525660] nix-grpc-daemon[1777]: time=2026-09-23T05:35:21.127Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6820lb # [ 201.466998] niks3-server[1137]: 2026/09/23 05:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign6821worker1 # [ 201.554433] nix-grpc-daemon[1777]: time=2026-09-23T05:35:21.158Z level=INFO msg="Uploading 1 narinfos"6822lb # [ 201.483076] niks3-server[1137]: 2026/09/23 05:35:21 INFO Signed narinfos id=86 count=16823lb # [ 201.489841] niks3-server[1137]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6824worker1 # [ 201.577414] nix-grpc-daemon[1777]: time=2026-09-23T05:35:21.181Z level=INFO msg="Upload complete. (215ms)"6825lb # [ 201.501101] niks3-server[1137]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6826worker1 # [ 201.584106] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=0 paths=1 nar_bytes_in=7526827lb # [ 201.510779] niks3-server[1137]: 2026/09/23 05:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete6828lb # [ 201.515107] niks3-server[1137]: 2026/09/23 05:35:21 INFO Completed upload id=866829worker1 # [ 201.593129] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478166830lb # [ 201.518283] niks3-server[1137]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6831worker1 # [ 201.603627] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6832worker1 # [ 201.608785] nix-daemon[854]: reaped child process 2290, status = succeeded6833worker1 # [ 201.621711] nix-daemon[854]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6834worker1 # [ 202.649585] nix-daemon[854]: reaped child process 2299, status = succeeded6835worker1 # [ 202.750965] nix-daemon[854]: accepted connection from pid 2315, user nix-grpc-daemon (trusted)6836worker1 # [ 202.767776] nix-daemon[854]: reaped child process 2318, status = succeeded6837lb # [ 202.698101] niks3-server[1137]: 2026/09/23 05:35:22 INFO Received uploads request method=POST path=/api/pending_closures6838worker1 # [ 202.785889] nix-grpc-daemon[1777]: time=2026-09-23T05:35:22.388Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6839worker1 # [ 202.787330] nix-grpc-daemon[1777]: time=2026-09-23T05:35:22.388Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6840lb # [ 202.738413] niks3-server[1137]: 2026/09/23 05:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign6841worker1 # [ 202.824945] nix-grpc-daemon[1777]: time=2026-09-23T05:35:22.428Z level=INFO msg="Uploading 1 narinfos"6842lb # [ 202.753777] niks3-server[1137]: 2026/09/23 05:35:22 INFO Signed narinfos id=87 count=16843lb # [ 202.760763] niks3-server[1137]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6844lb # [ 202.767808] niks3-server[1137]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6845lb # [ 202.777778] niks3-server[1137]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6846worker1 # [ 202.859347] nix-grpc-daemon[1777]: time=2026-09-23T05:35:22.463Z level=INFO msg="Upload complete. (229ms)"6847lb # [ 202.783591] niks3-server[1137]: 2026/09/23 05:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete6848lb # [ 202.787347] niks3-server[1137]: 2026/09/23 05:35:22 INFO Completed upload id=876849lb # [ 202.790911] niks3-server[1137]: 2026/09/23 05:35:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6850worker1 # [ 202.873417] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47816 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=16851worker1 # [ 202.887835] nix-daemon[854]: reaped child process 2297, status = succeeded6852client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.28 seconds)6853worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6854worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6855client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6856client # [ 202.991106] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6857client # Running as unit: gpu.service; invocation ID: 93639ee65da94e75b2455bd0d7a37e516858client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6859lb: must succeed: curl -sf localhost:9901/clusters6860lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6861lb: must succeed: curl -sf localhost:9901/clusters6862lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6863worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6864client # [ 203.193174] nix[1624]: don't know how to build these paths:6865client # [ 203.194201] nix[1624]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6866worker1 # [ 203.268966] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478166867worker2 # [ 203.206930] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331006868worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6869worker1 # [ 203.301864] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:22Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06870client # [ 203.231683] nix[1624]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6871lb: must succeed: curl -sf localhost:9901/clusters6872lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6873lb: must succeed: curl -sf localhost:9901/clusters6874lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6875worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6876worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6877client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6878lb # [ 209.633359] envoy[904]: conn peer=[2001:db8:1::1]:47776 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27123 tx=12319 ms=150036879client # [ 233.236872] nix[1624]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6880client # [ 233.250371] nix[1624]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6881client # [ 233.290222] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6882client # [ 233.298478] systemd[1]: gpu.service: Failed with result 'exit-code'.6883client # [ 233.302353] systemd[1]: gpu.service: Consumed 202ms CPU time over 30.304s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6884client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.06 seconds)6885client: must succeed: journalctl -u gpu -o cat6886client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6887lb: must succeed: curl -sf localhost:9901/clusters6888lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6889lb: must succeed: curl -sf localhost:9901/clusters6890lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6891worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6892worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6893(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.07 seconds)6894subtest: low disk drains a worker and builds go to the other6895worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6896worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.20 seconds)6897worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6898worker1 # [ 235.335692] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:54Z level=info event=unhealthy reason=min_free6899worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6900client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26901client # don't know how to build these paths:6902client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6903worker2 # [ 236.174584] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:331006904worker2 # [ 236.183202] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331006905worker1 # [ 236.275212] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:55Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06906worker1 # [ 236.275619] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:55Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16907worker2 # [ 236.250659] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331006908worker2 # [ 236.265977] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331006909client # copying 1 paths to 'lb:50051'...6910client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6911worker2 # [ 236.281830] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331006912worker2 # [ 236.296977] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6913worker2 # [ 236.460163] nix-daemon[839]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6914worker2 # [ 236.482500] nix-daemon[839]: reaped child process 1934, status = succeeded6915lb # [ 236.477745] niks3-server[1137]: 2026/09/23 05:35:56 INFO Received uploads request method=POST path=/api/pending_closures6916worker2 # [ 236.496414] nix-grpc-daemon[1836]: time=2026-09-23T05:35:56.111Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6917worker2 # [ 236.498306] nix-grpc-daemon[1836]: time=2026-09-23T05:35:56.112Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6918lb # [ 236.514972] niks3-server[1137]: 2026/09/23 05:35:56 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6919worker2 # [ 236.546930] nix-grpc-daemon[1836]: time=2026-09-23T05:35:56.162Z level=INFO msg="Uploading 1 narinfos"6920lb # [ 236.533978] niks3-server[1137]: 2026/09/23 05:35:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign6921lb # [ 236.544278] niks3-server[1137]: 2026/09/23 05:35:56 INFO Signed narinfos id=88 count=16922lb # [ 236.556122] niks3-server[1137]: 2026/09/23 05:35:56 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete6923lb # [ 236.569891] niks3-server[1137]: 2026/09/23 05:35:56 INFO Completed upload id=886924worker2 # [ 236.575751] nix-grpc-daemon[1836]: time=2026-09-23T05:35:56.192Z level=INFO msg="Upload complete. (261ms)"6925lb # [ 236.576070] niks3-server[1137]: 2026/09/23 05:35:56 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6926worker2 # [ 236.588655] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=5126927lb # [ 236.589588] niks3-server[1137]: 2026/09/23 05:35:56 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6928worker2 # [ 236.600560] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331006929worker2 # [ 236.615673] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6930worker2 # [ 236.624187] nix-daemon[839]: reaped child process 1929, status = succeeded6931client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6932worker2 # [ 236.632073] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6933client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6934client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6935client # farm-a> LOG-farm-a-drain6936worker2 # [ 237.500806] nix-daemon[839]: reaped child process 1938, status = succeeded6937worker2 # [ 237.605603] nix-daemon[839]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)6938worker2 # [ 237.623436] nix-daemon[839]: reaped child process 1957, status = succeeded6939lb # [ 237.624815] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received uploads request method=POST path=/api/pending_closures6940worker2 # [ 237.643757] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.258Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6941worker2 # [ 237.645639] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.258Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6942lb # [ 237.662644] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6943lb # [ 237.676489] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6944worker2 # [ 237.691123] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.307Z level=INFO msg="Uploading 1 narinfos"6945lb # [ 237.685011] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign6946lb # [ 237.693477] niks3-server[1137]: 2026/09/23 05:35:57 INFO Signed narinfos id=89 count=16947lb # [ 237.697942] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6948lb # [ 237.708314] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete6949lb # [ 237.715066] niks3-server[1137]: 2026/09/23 05:35:57 INFO Completed upload id=896950lb # [ 237.717651] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6951worker2 # [ 237.720859] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.337Z level=INFO msg="Upload complete. (279ms)"6952worker2 # [ 237.733290] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=16953worker1 # [ 237.808504] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:57Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06954worker1 # [ 237.808789] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:57Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16955worker2 # [ 237.755006] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331006956worker2 # [ 237.761694] nix-daemon[839]: reaped child process 1936, status = succeeded6957worker2 # [ 237.769000] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331006958worker2 # [ 237.781178] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.397Z level=INFO msg="All 1 paths already cached"6959client # copying 1 paths to 'lb:50051'...6960client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6961worker2 # [ 237.791601] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331006962worker2 # [ 237.807368] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6963worker2 # [ 237.947482] nix-daemon[839]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)6964worker2 # [ 237.963805] nix-daemon[839]: reaped child process 1964, status = succeeded6965lb # [ 237.964242] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received uploads request method=POST path=/api/pending_closures6966worker2 # [ 237.980087] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.596Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6967worker2 # [ 237.981578] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.596Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6968lb # [ 237.992268] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign6969worker2 # [ 238.006146] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.622Z level=INFO msg="Uploading 1 narinfos"6970lb # [ 238.006660] niks3-server[1137]: 2026/09/23 05:35:57 INFO Signed narinfos id=90 count=16971lb # [ 238.013579] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6972lb # [ 238.024905] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6973worker2 # [ 238.038047] nix-grpc-daemon[1836]: time=2026-09-23T05:35:57.654Z level=INFO msg="Upload complete. (219ms)"6974lb # [ 238.031765] niks3-server[1137]: 2026/09/23 05:35:57 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete6975lb # [ 238.038738] niks3-server[1137]: 2026/09/23 05:35:57 INFO Completed upload id=906976lb # [ 238.042950] niks3-server[1137]: 2026/09/23 05:35:57 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6977worker2 # [ 238.044497] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=6326978worker2 # [ 238.057866] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331006979worker2 # [ 238.066695] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6980worker2 # [ 238.069110] nix-daemon[839]: reaped child process 1959, status = succeeded6981client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6982worker2 # [ 238.081545] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6983client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6984client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6985client # farm-b> LOG-farm-b-drain6986worker2 # [ 239.306789] nix-daemon[839]: reaped child process 1968, status = succeeded6987worker2 # [ 239.431353] nix-daemon[839]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)6988worker2 # [ 239.450458] nix-daemon[839]: reaped child process 1987, status = succeeded6989lb # [ 239.449327] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received uploads request method=POST path=/api/pending_closures6990worker2 # [ 239.470291] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.084Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6991worker2 # [ 239.471989] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.084Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6992lb # [ 239.493540] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign6993worker2 # [ 239.508768] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.124Z level=INFO msg="Uploading 1 narinfos"6994lb # [ 239.509035] niks3-server[1137]: 2026/09/23 05:35:59 INFO Signed narinfos id=91 count=16995lb # [ 239.515055] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6996lb # [ 239.525250] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6997worker2 # [ 239.537445] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.153Z level=INFO msg="Upload complete. (246ms)"6998lb # [ 239.535857] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete6999lb # [ 239.542149] niks3-server[1137]: 2026/09/23 05:35:59 INFO Completed upload id=917000worker2 # [ 239.549384] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=17001lb # [ 239.545713] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7002lb # [ 239.549841] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7003worker2 # [ 239.566285] nix-daemon[839]: reaped child process 1966, status = succeeded7004worker1 # [ 239.635358] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:59Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07005worker1 # [ 239.642199] nix-grpc-daemon[1696]: ts=2026-09-23T05:35:59Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17006worker2 # [ 239.611184] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331007007worker2 # [ 239.626223] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331007008lb # [ 239.634699] envoy[904]: conn peer=[2001:db8:1::1]:38154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300707009worker2 # [ 239.647384] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.263Z level=INFO msg="All 2 paths already cached"7010client # copying 1 paths to 'lb:50051'...7011client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7012worker2 # [ 239.658825] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331007013worker2 # [ 239.672118] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)7014worker2 # [ 239.834976] nix-daemon[839]: accepted connection from pid 1991, user nix-grpc-daemon (trusted)7015worker2 # [ 239.853599] nix-daemon[839]: reaped child process 1994, status = succeeded7016lb # [ 239.850182] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received uploads request method=POST path=/api/pending_closures7017worker2 # [ 239.865581] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.481Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7018worker2 # [ 239.867074] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.481Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7019lb # [ 239.879590] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7020worker2 # [ 239.898335] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.514Z level=INFO msg="Uploading 1 narinfos"7021lb # [ 239.894320] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7022lb # [ 239.904241] niks3-server[1137]: 2026/09/23 05:35:59 INFO Signed narinfos id=92 count=17023worker2 # [ 239.918488] nix-grpc-daemon[1836]: time=2026-09-23T05:35:59.534Z level=INFO msg="Upload complete. (228ms)"7024lb # [ 239.913906] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7025worker2 # [ 239.927093] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=0 paths=1 nar_bytes_in=7607026lb # [ 239.922466] niks3-server[1137]: 2026/09/23 05:35:59 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7027lb # [ 239.926201] niks3-server[1137]: 2026/09/23 05:35:59 INFO Completed upload id=927028lb # [ 239.928955] niks3-server[1137]: 2026/09/23 05:35:59 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7029worker2 # [ 239.938956] nix-grpc-daemon[1775]: ts=2026-09-23T05:35:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331007030worker2 # [ 239.947892] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)7031worker2 # [ 239.951019] nix-daemon[839]: reaped child process 1989, status = succeeded7032client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7033worker2 # [ 239.961508] nix-daemon[839]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)7034client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7035client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7036client # farm-top> LOG-farm-top-drain7037worker2 # [ 241.193214] nix-daemon[839]: reaped child process 1998, status = succeeded7038worker2 # [ 241.295190] nix-daemon[839]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)7039worker2 # [ 241.315889] nix-daemon[839]: reaped child process 2017, status = succeeded7040lb # [ 241.313952] niks3-server[1137]: 2026/09/23 05:36:00 INFO Received uploads request method=POST path=/api/pending_closures7041worker2 # [ 241.332385] nix-grpc-daemon[1836]: time=2026-09-23T05:36:00.946Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7042worker2 # [ 241.333823] nix-grpc-daemon[1836]: time=2026-09-23T05:36:00.946Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7043lb # [ 241.356442] niks3-server[1137]: 2026/09/23 05:36:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7044worker2 # [ 241.371445] nix-grpc-daemon[1836]: time=2026-09-23T05:36:00.987Z level=INFO msg="Uploading 1 narinfos"7045lb # [ 241.369662] niks3-server[1137]: 2026/09/23 05:36:00 INFO Signed narinfos id=93 count=17046lb # [ 241.378578] niks3-server[1137]: 2026/09/23 05:36:00 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7047lb # [ 241.387353] niks3-server[1137]: 2026/09/23 05:36:00 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7048lb # [ 241.396756] niks3-server[1137]: 2026/09/23 05:36:00 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7049worker2 # [ 241.408603] nix-grpc-daemon[1836]: time=2026-09-23T05:36:01.024Z level=INFO msg="Upload complete. (235ms)"7050lb # [ 241.402490] niks3-server[1137]: 2026/09/23 05:36:00 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7051lb # [ 241.407024] niks3-server[1137]: 2026/09/23 05:36:01 INFO Completed upload id=937052lb # [ 241.413947] niks3-server[1137]: 2026/09/23 05:36:01 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7053worker2 # [ 241.421060] nix-grpc-daemon[1775]: ts=2026-09-23T05:36:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33100 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=17054worker2 # [ 241.436761] nix-daemon[839]: reaped child process 1996, status = succeeded7055client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.51 seconds)7056client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7057client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7058worker1: must succeed: rm /nix/.rw-store/fill7059worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7060worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7061worker1 # [ 242.339560] nix-grpc-daemon[1696]: ts=2026-09-23T05:36:01Z level=info event=healthy reason=min_free7062worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7063(finished: subtest: low disk drains a worker and builds go to the other, in 9.31 seconds)7064(finished: run the VM test script, in 243.87 seconds)7065test script finished in 244.11s7066cleanup7067kill QemuMachine (pid 49)7068client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7069client # [2026-09-23T05:36:02Z INFO virtiofsd] Client disconnected, shutting down7070client # [2026-09-23T05:36:02Z INFO virtiofsd] Client disconnected, shutting down7071client # [2026-09-23T05:36:02Z INFO virtiofsd] Client disconnected, shutting down7072kill QemuMachine (pid 50)7073lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7074lb # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7075lb # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7076lb # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7077kill QemuMachine (pid 51)7078worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7079worker1 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7080worker1 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7081worker1 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7082kill QemuMachine (pid 52)7083worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7084worker2 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7085worker2 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7086worker2 # [2026-09-23T05:36:03Z INFO virtiofsd] Client disconnected, shutting down7087(finished: cleanup, in 1.03 seconds)7088additionally exposed symbols:7089 client, lb, worker1, worker2,7090 vlan1,7091 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh