vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #545
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14client: QEMU running (pid 50)15worker2 # Disk image does not exist, creating the virtualisation disk image...16client # Formatting '/build/vm-state-client/tmp.Vbt7kasYn5', fmt=raw size=107374182417lb # Formatting '/build/vm-state-lb/tmp.ydSiXrbfBR', fmt=raw size=107374182418lb: QEMU running (pid 49)19worker2 # Formatting '/build/vm-state-worker2/tmp.jQJ4aTZlx7', fmt=raw size=107374182420client # mke2fs 1.47.4 (6-Mar-2025)21lb # mke2fs 1.47.4 (6-Mar-2025)22worker2: QEMU running (pid 52)23worker1: QEMU running (pid 51)24worker1 # Disk image does not exist, creating the virtualisation disk image...25worker2 # mke2fs 1.47.4 (6-Mar-2025)26lb # Discarding device blocks: 0/262144 done27client # Discarding device blocks: 0/262144 done28(finished: start all VMs, in 0.47 seconds)29lb # Creating filesystem with 262144 4k blocks and 65536 inodes30client # Creating filesystem with 262144 4k blocks and 65536 inodes31client: waiting for unit network-addresses-eth1.service32lb # Filesystem UUID: 35c0c8a8-d05a-4b64-9b2d-7848470362e433client # Filesystem UUID: ba44cbe0-3bc0-4136-a1b4-f5cba478abd434client: waiting for the VM to finish booting35lb # Superblock backups stored on blocks:36client # Superblock backups stored on blocks:37worker2 # Discarding device blocks: 0/262144 done38worker1 # Formatting '/build/vm-state-worker1/tmp.HnfbNDYmRD', fmt=raw size=107374182439lb # 32768, 98304, 163840, 22937640client # 32768, 98304, 163840, 22937641worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes42worker1 # mke2fs 1.47.4 (6-Mar-2025)43lb # 44client # 45worker2 # Filesystem UUID: 1aa51b4b-307d-4b5b-add0-1319d722241646worker1 # Discarding device blocks: 0/262144 done47lb # Allocating group tables: 0/8 done48client # Allocating group tables: 0/8 done49worker2 # Superblock backups stored on blocks:50worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes51lb # Writing inode tables: 0/8 done52client # Writing inode tables: 0/8 done53worker2 # 32768, 98304, 163840, 22937654worker1 # Filesystem UUID: fa43526a-b7b9-4ce7-94fc-25384db8b3b755lb # Creating journal (8192 blocks): done56client # Creating journal (8192 blocks): done57worker2 # 58worker1 # Superblock backups stored on blocks:59lb # Writing superblocks and filesystem accounting information: 0/8 done60client # Writing superblocks and filesystem accounting information: 0/8 done61worker2 # Allocating group tables: 0/8 done62worker1 # 32768, 98304, 163840, 22937663lb # 64client # 65worker2 # Writing inode tables: 0/8 done66worker1 # 67lb # Virtualisation disk image created.68client # Virtualisation disk image created.69worker2 # Creating journal (8192 blocks): done70worker1 # Allocating group tables: 0/8 done71lb # Starting virtiofs daemons...72client # Starting virtiofs daemons...73worker2 # Writing superblocks and filesystem accounting information: 0/8 done74worker1 # Writing inode tables: 0/8 done75lb # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76client # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77worker2 # 78worker1 # Creating journal (8192 blocks): done79lb # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether80client # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether81worker2 # Virtualisation disk image created.82worker1 # Writing superblocks and filesystem accounting information: 0/8 done83lb # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...84client # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...85worker2 # Starting virtiofs daemons...86worker1 # 87lb # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88client # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)89worker2 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90worker1 # Virtualisation disk image created.91lb # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92client # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93worker2 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether94worker1 # Starting virtiofs daemons...95lb # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...96client # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...97worker2 # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...98worker1 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)99lb # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100client # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)101worker2 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)102worker1 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether103lb # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104client # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether105worker2 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether106worker1 # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...107lb # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...108client # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...109worker2 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)110worker1 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)111lb # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests112client # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests113worker2 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether114worker1 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether115lb # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests116client # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests117worker2 # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...118worker1 # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...119lb # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests120client # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests121worker2 # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker2 # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests124worker1 # [2026-09-23T10:34:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether125worker2 # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests126worker1 # [2026-09-23T10:34:12Z INFO virtiofsd] Waiting for vhost-user socket connection...127worker2 # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests128worker1 # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-23T10:34:12Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker1 # [ 0.000000] KASLR enabled134worker1 # [ 0.000000] random: crng init done135worker1 # [ 0.000000] Machine model: linux,dummy-virt136worker1 # [ 0.000000] efi: UEFI not found.137worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140worker1 # [ 0.000000] Zone ranges:141worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142worker1 # [ 0.000000] DMA32 empty143worker1 # [ 0.000000] Normal empty144worker1 # [ 0.000000] Device empty145worker1 # [ 0.000000] Movable zone start for each node146worker1 # [ 0.000000] Early memory node ranges147worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker1 # [ 0.000000] psci: probing for conduit method from DT.151worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153worker1 # [ 0.000000] psci: Trusted OS migration not required154worker1 # [ 0.000000] psci: SMC Calling Convention v1.1155worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157worker1 # [ 0.000000] Detected PIPT I-cache on CPU0158worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface160worker1 # [ 0.000000] CPU features: detected: Spectre-v4161worker1 # [ 0.000000] CPU features: detected: Spectre-BHB162worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164worker1 # [ 0.000000] alternatives: applying boot alternatives165worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hpnyxc8hxy8zrxyq3671sa25qlfia9l8-nixos-system-worker1-test/init regInfo=/nix/store/rhd0rniq002jqnsgj86gr09d6bnymc39-closure-info/registration console=ttyAMA0,115200n8 console=tty0166worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/rhd0rniq002jqnsgj86gr09d6bnymc39-closure-info/registration", will be passed to user space.167worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)169worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)170worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB171worker1 # [ 0.000000] software IO TLB: area num 1.172worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB173worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)174worker1 # [ 0.000000] Fallback order for Node 0: 0175worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216176worker1 # [ 0.000000] Policy zone: DMA177worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off178worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1179worker1 # [ 0.000000] allocated 3145728 bytes of page_ext180worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages181worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups182worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.183worker1 # [ 0.000000] rcu: RCU event tracing is enabled.184worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.185worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.186worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.187worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.188worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.189worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1190worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.191worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.192worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.193worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0194worker1 # [ 0.000000] GICv3: 256 SPIs implemented195worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented196worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq197worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI198worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0199worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000200worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]201worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)202worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)203worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000204worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000205worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.206worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns207worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).208worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns209worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]210lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]211worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns212worker1 # [ 0.000035] arm-pv: using stolen time PV213worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026214lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026215worker2 # [ 0.000000] KASLR enabled216worker1 # [ 0.000663] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)217lb # [ 0.000000] KASLR enabled218worker2 # [ 0.000000] random: crng init done219lb # [ 0.000000] random: crng init done220worker1 # [ 0.000827] Console: colour dummy device 80x25221worker2 # [ 0.000000] Machine model: linux,dummy-virt222lb # [ 0.000000] Machine model: linux,dummy-virt223worker2 # [ 0.000000] efi: UEFI not found.224worker1 # [ 0.000834] printk: legacy console [tty0] enabled225lb # [ 0.000000] efi: UEFI not found.226client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]227worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT228lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT229worker1 # [ 0.001049] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)230worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]231worker1 # [ 0.001056] pid_max: default: 32768 minimum: 301232client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026233lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]234client # [ 0.000000] KASLR enabled235worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]236worker1 # [ 0.001136] LSM: initializing lsm=capability,landlock,yama,bpf,ima237lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]238client # [ 0.000000] random: crng init done239worker2 # [ 0.000000] Zone ranges:240lb # [ 0.000000] Zone ranges:241worker1 # [ 0.001323] landlock: Up and running.242client # [ 0.000000] Machine model: linux,dummy-virt243worker1 # [ 0.001326] Yama: becoming mindful.244worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]245client # [ 0.000000] efi: UEFI not found.246lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]247worker2 # [ 0.000000] DMA32 empty248worker1 # [ 0.001995] LSM support for eBPF active249lb # [ 0.000000] DMA32 empty250worker2 # [ 0.000000] Normal empty251lb # [ 0.000000] Normal empty252client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT253worker2 # [ 0.000000] Device empty254lb # [ 0.000000] Device empty255worker1 # [ 0.002164] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)256worker2 # [ 0.000000] Movable zone start for each node257client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]258worker2 # [ 0.000000] Early memory node ranges259lb # [ 0.000000] Movable zone start for each node260worker1 # [ 0.002202] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)261lb # [ 0.000000] Early memory node ranges262client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]263worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]264worker1 # [ 0.003918] rcu: Hierarchical SRCU implementation.265client # [ 0.000000] Zone ranges:266lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]267worker1 # [ 0.003923] rcu: Max phase no-delay instances is 1000.268worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]269client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]270lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]271client # [ 0.000000] DMA32 empty272worker1 # [ 0.005150] fsl-mc MSI: its@8080000 domain created273worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000274client # [ 0.000000] Normal empty275lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000276worker1 # [ 0.005250] EFI services will not be available.277client # [ 0.000000] Device empty278worker2 # [ 0.000000] psci: probing for conduit method from DT.279lb # [ 0.000000] psci: probing for conduit method from DT.280client # [ 0.000000] Movable zone start for each node281worker1 # [ 0.005325] smp: Bringing up secondary CPUs ...282worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.283lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.284client # [ 0.000000] Early memory node ranges285worker1 # [ 0.005333] smp: Brought up 1 node, 1 CPU286worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs287lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs288worker1 # [ 0.005336] SMP: Total of 1 processors activated.289client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]290worker2 # [ 0.000000] psci: Trusted OS migration not required291worker1 # [ 0.005342] CPU: All CPU(s) started at EL1292lb # [ 0.000000] psci: Trusted OS migration not required293worker2 # [ 0.000000] psci: SMC Calling Convention v1.1294client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]295lb # [ 0.000000] psci: SMC Calling Convention v1.1296worker1 # [ 0.005352] CPU features: detected: Branch Target Identification297client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000298worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)299worker1 # [ 0.005357] CPU features: detected: ARMv8.4 Translation Table Level300lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)301client # [ 0.000000] psci: probing for conduit method from DT.302worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296303client # [ 0.000000] psci: PSCIv1.3 detected in firmware.304lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296305worker1 # [ 0.005360] CPU features: detected: Instruction cache invalidation not required for I/D coherence306worker2 # [ 0.000000] Detected PIPT I-cache on CPU0307lb # [ 0.000000] Detected PIPT I-cache on CPU0308client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs309client # [ 0.000000] psci: Trusted OS migration not required310worker1 # [ 0.005364] CPU features: detected: Data cache clean to the PoU not required for I/D coherence311worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)312lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)313client # [ 0.000000] psci: SMC Calling Convention v1.1314worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface315worker1 # [ 0.005367] CPU features: detected: Common not Private translations316lb # [ 0.000000] CPU features: detected: GICv3 CPU interface317worker2 # [ 0.000000] CPU features: detected: Spectre-v4318worker1 # [ 0.005371] CPU features: detected: CRC32 instructions319lb # [ 0.000000] CPU features: detected: Spectre-v4320client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)321worker2 # [ 0.000000] CPU features: detected: Spectre-BHB322lb # [ 0.000000] CPU features: detected: Spectre-BHB323worker1 # [ 0.005374] CPU features: detected: Data cache clean to Point of Deep Persistence324client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296325worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38326lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38327client # [ 0.000000] Detected PIPT I-cache on CPU0328worker1 # [ 0.005377] CPU features: detected: Data cache clean to Point of Persistence329worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23330lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23331client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)332worker2 # [ 0.000000] alternatives: applying boot alternatives333worker1 # [ 0.005380] CPU features: detected: Data independent timing control (DIT)334lb # [ 0.000000] alternatives: applying boot alternatives335client # [ 0.000000] CPU features: detected: GICv3 CPU interface336client # [ 0.000000] CPU features: detected: Spectre-v4337worker1 # [ 0.005383] CPU features: detected: E0PD338client # [ 0.000000] CPU features: detected: Spectre-BHB339worker1 # [ 0.005386] CPU features: detected: Enhanced Counter Virtualization340client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38341worker1 # [ 0.005389] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)342client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23343client # [ 0.000000] alternatives: applying boot alternatives344worker1 # [ 0.005392] CPU features: detected: Enhanced Virtualization Traps345worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ww1604hqspfdnpd2s7q2dqyp5z5n4fq8-nixos-system-worker2-test/init regInfo=/nix/store/pwb1gai0j7b5i1qi36sgpa6szrlksbpv-closure-info/registration console=ttyAMA0,115200n8 console=tty0346lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3in2qynk4v72bmzzvgahb746q83gs855-nixos-system-lb-test/init regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration console=ttyAMA0,115200n8 console=tty0347worker1 # [ 0.005395] CPU features: detected: Fine Grained Traps348worker1 # [ 0.005399] CPU features: detected: Generic authentication (architected QARMA5 algorithm)349worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/pwb1gai0j7b5i1qi36sgpa6szrlksbpv-closure-info/registration", will be passed to user space.350lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration", will be passed to user space.351worker1 # [ 0.005403] CPU features: detected: RCpc load-acquire (LDAPR)352worker1 # [ 0.005406] CPU features: detected: LSE atomic instructions353worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes354lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes355worker1 # [ 0.005409] CPU features: detected: Privileged Access Never356client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ra6r7v42kn1phhf50aiw8qw3p1s4g3i7-nixos-system-client-test/init regInfo=/nix/store/lgfd5irrpwicr30w9sv4rcyvvpw4p2ws-closure-info/registration console=ttyAMA0,115200n8 console=tty0357worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)358lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)359worker1 # [ 0.005412] CPU features: detected: PMUv3360worker1 # [ 0.005415] CPU features: detected: RAS Extension Support361worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)362lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)363worker1 # [ 0.005418] CPU features: detected: RASv1p1 Extension Support364client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lgfd5irrpwicr30w9sv4rcyvvpw4p2ws-closure-info/registration", will be passed to user space.365worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB366lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB367worker2 # [ 0.000000] software IO TLB: area num 1.368worker1 # [ 0.005420] CPU features: detected: Random Number Generator369lb # [ 0.000000] software IO TLB: area num 1.370client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes371worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB372worker1 # [ 0.005423] CPU features: detected: Speculation barrier (SB)373lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB374client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)375worker1 # [ 0.005425] CPU features: detected: Stage-2 Force Write-Back376worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)377lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)378worker1 # [ 0.005428] CPU features: detected: TLB range maintenance instructions379client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)380lb # [ 0.000000] Fallback order for Node 0: 0381worker2 # [ 0.000000] Fallback order for Node 0: 0382client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB383worker1 # [ 0.005433] CPU features: detected: Speculative Store Bypassing Safe (SSBS)384lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216385worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216386lb # [ 0.000000] Policy zone: DMA387worker2 # [ 0.000000] Policy zone: DMA388client # [ 0.000000] software IO TLB: area num 1.389worker1 # [ 0.005471] alternatives: applying system-wide alternatives390worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off391lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off392client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB393worker1 # [ 0.008362] CPU features: detected: BBM Level 2 without TLB conflict abort394worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1395lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1396client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)397worker2 # [ 0.000000] allocated 3145728 bytes of page_ext398lb # [ 0.000000] allocated 3145728 bytes of page_ext399client # [ 0.000000] Fallback order for Node 0: 0400lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages401worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages402worker1 # [ 0.008568] Memory: 1405884K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)403client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216404worker1 # [ 0.008939] devtmpfs: initialized405worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups406lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups407client # [ 0.000000] Policy zone: DMA408lb # [ 0.000000] rcu: Hierarchical RCU implementation.409worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.410worker1 # [ 0.010781] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)411client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off412lb # [ 0.000000] rcu: RCU event tracing is enabled.413worker2 # [ 0.000000] rcu: RCU event tracing is enabled.414client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1415worker1 # [ 0.010803] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).416worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.417lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.418client # [ 0.000000] allocated 3145728 bytes of page_ext419worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.420lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.421worker1 # [ 0.010981] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL422client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages423worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.424lb # [ 0.000000] Rude variant of Tasks RCU enabled.425worker1 # [ 0.010986] 0 pages in range for non-PLT usage426client # [ 0.000000] ftrace: allocated 294 pages with 4 groups427lb # [ 0.000000] Tracing variant of Tasks RCU enabled.428worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.429worker1 # [ 0.010987] 508272 pages in range for PLT usage430client # [ 0.000000] rcu: Hierarchical RCU implementation.431worker1 # [ 0.011094] pinctrl core: initialized pinctrl subsystem432worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.433lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.434client # [ 0.000000] rcu: RCU event tracing is enabled.435worker1 # [ 0.011914] DMI not present or invalid.436worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1437client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.438worker1 # [ 0.015686] NET: Registered PF_NETLINK/PF_ROUTE protocol family439lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1440client # [ 0.000000] Trampoline variant of Tasks RCU enabled.441worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.442worker1 # [ 0.017879] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations443lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.444client # [ 0.000000] Rude variant of Tasks RCU enabled.445client # [ 0.000000] Tracing variant of Tasks RCU enabled.446worker1 # [ 0.018062] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations447lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.448worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.449client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.450worker1 # [ 0.018267] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations451worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.452lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.453client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1454worker1 # [ 0.018288] audit: initializing netlink subsys (disabled)455lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0456worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0457worker1 # [ 0.018822] thermal_sys: Registered thermal governor 'fair_share'458lb # [ 0.000000] GICv3: 256 SPIs implemented459worker2 # [ 0.000000] GICv3: 256 SPIs implemented460client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.461lb # [ 0.000000] GICv3: 0 Extended SPIs implemented462worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented463worker1 # [ 0.018824] thermal_sys: Registered thermal governor 'bang_bang'464lb # [ 0.000000] Root IRQ handler: gic_handle_irq465worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq466client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.467worker1 # [ 0.018828] thermal_sys: Registered thermal governor 'step_wise'468lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI469worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI470lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0471worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0472worker1 # [ 0.018830] thermal_sys: Registered thermal governor 'user_space'473client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.474client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0475worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000476worker1 # [ 0.018836] thermal_sys: Registered thermal governor 'power_allocator'477lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000478lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]479client # [ 0.000000] GICv3: 256 SPIs implemented480worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]481worker1 # [ 0.018861] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1482client # [ 0.000000] GICv3: 0 Extended SPIs implemented483worker1 # [ 0.018869] cpuidle: using governor ladder484client # [ 0.000000] Root IRQ handler: gic_handle_irq485lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)486worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)487worker1 # [ 0.018875] cpuidle: using governor menu488client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI489worker1 # [ 0.019074] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.490client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0491worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)492lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)493worker1 # [ 0.019090] ASID allocator initialised with 65536 entries494lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000495worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000496client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000497worker1 # [ 0.020391] Serial: AMBA PL011 UART driver498client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]499lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000500worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000501worker1 # [ 0.025815] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1502lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.503worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.504client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)505worker1 # [ 0.025965] printk: console [ttyAMA0] enabled506lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns507worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns508client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)509lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).510worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).511client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000512client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000513lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns514worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns515client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.516lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns517worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns518lb # [ 0.000029] arm-pv: using stolen time PV519worker2 # [ 0.000028] arm-pv: using stolen time PV520client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns521client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).522lb # [ 0.000380] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)523worker2 # [ 0.000397] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)524lb # [ 0.000536] Console: colour dummy device 80x25525worker2 # [ 0.000572] Console: colour dummy device 80x25526client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns527lb # [ 0.000543] printk: legacy console [tty0] enabled528worker2 # [ 0.000580] printk: legacy console [tty0] enabled529worker1 # [ 0.145775] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages530client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns531worker1 # [ 0.145791] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page532lb # [ 0.000756] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)533worker2 # [ 0.000766] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)534client # [ 0.000029] arm-pv: using stolen time PV535worker1 # [ 0.145796] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages536worker2 # [ 0.000773] pid_max: default: 32768 minimum: 301537lb # [ 0.000762] pid_max: default: 32768 minimum: 301538lb # [ 0.000841] LSM: initializing lsm=capability,landlock,yama,bpf,ima539worker1 # [ 0.145801] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page540worker2 # [ 0.000841] LSM: initializing lsm=capability,landlock,yama,bpf,ima541client # [ 0.000407] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)542lb # [ 0.000983] landlock: Up and running.543worker2 # [ 0.000961] landlock: Up and running.544client # [ 0.000578] Console: colour dummy device 80x25545lb # [ 0.000985] Yama: becoming mindful.546worker1 # [ 0.145806] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages547worker2 # [ 0.000963] Yama: becoming mindful.548client # [ 0.000585] printk: legacy console [tty0] enabled549lb # [ 0.001408] LSM support for eBPF active550worker2 # [ 0.001417] LSM support for eBPF active551worker1 # [ 0.145810] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page552lb # [ 0.001540] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)553worker2 # [ 0.001548] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)554client # [ 0.000779] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)555worker1 # [ 0.145814] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages556client # [ 0.000786] pid_max: default: 32768 minimum: 301557lb # [ 0.001578] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)558worker2 # [ 0.001582] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)559worker1 # [ 0.145819] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page560lb # [ 0.003338] rcu: Hierarchical SRCU implementation.561client # [ 0.000856] LSM: initializing lsm=capability,landlock,yama,bpf,ima562worker2 # [ 0.003422] rcu: Hierarchical SRCU implementation.563client # [ 0.000982] landlock: Up and running.564lb # [ 0.003343] rcu: Max phase no-delay instances is 1000.565worker2 # [ 0.003428] rcu: Max phase no-delay instances is 1000.566client # [ 0.000985] Yama: becoming mindful.567lb # [ 0.004564] fsl-mc MSI: its@8080000 domain created568worker2 # [ 0.004653] fsl-mc MSI: its@8080000 domain created569client # [ 0.001404] LSM support for eBPF active570lb # [ 0.004662] EFI services will not be available.571worker1 # [ 0.152994] fbcon: Taking over console572worker2 # [ 0.004742] EFI services will not be available.573lb # [ 0.004733] smp: Bringing up secondary CPUs ...574client # [ 0.001550] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)575worker2 # [ 0.004828] smp: Bringing up secondary CPUs ...576worker1 # [ 0.153007] ACPI: Interpreter disabled.577lb # [ 0.004741] smp: Brought up 1 node, 1 CPU578worker2 # [ 0.004836] smp: Brought up 1 node, 1 CPU579client # [ 0.001582] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)580lb # [ 0.004744] SMP: Total of 1 processors activated.581worker2 # [ 0.004839] SMP: Total of 1 processors activated.582client # [ 0.003366] rcu: Hierarchical SRCU implementation.583lb # [ 0.004749] CPU: All CPU(s) started at EL1584worker2 # [ 0.004844] CPU: All CPU(s) started at EL1585client # [ 0.003370] rcu: Max phase no-delay instances is 1000.586lb # [ 0.004759] CPU features: detected: Branch Target Identification587worker2 # [ 0.004854] CPU features: detected: Branch Target Identification588client # [ 0.004593] fsl-mc MSI: its@8080000 domain created589lb # [ 0.004763] CPU features: detected: ARMv8.4 Translation Table Level590worker2 # [ 0.004859] CPU features: detected: ARMv8.4 Translation Table Level591client # [ 0.004680] EFI services will not be available.592client # [ 0.004795] smp: Bringing up secondary CPUs ...593lb # [ 0.004766] CPU features: detected: Instruction cache invalidation not required for I/D coherence594worker2 # [ 0.004862] CPU features: detected: Instruction cache invalidation not required for I/D coherence595client # [ 0.004803] smp: Brought up 1 node, 1 CPU596client # [ 0.004806] SMP: Total of 1 processors activated.597lb # [ 0.004770] CPU features: detected: Data cache clean to the PoU not required for I/D coherence598worker2 # [ 0.004866] CPU features: detected: Data cache clean to the PoU not required for I/D coherence599client # [ 0.004811] CPU: All CPU(s) started at EL1600lb # [ 0.004774] CPU features: detected: Common not Private translations601worker2 # [ 0.004870] CPU features: detected: Common not Private translations602client # [ 0.004821] CPU features: detected: Branch Target Identification603lb # [ 0.004778] CPU features: detected: CRC32 instructions604worker2 # [ 0.004873] CPU features: detected: CRC32 instructions605client # [ 0.004825] CPU features: detected: ARMv8.4 Translation Table Level606lb # [ 0.004780] CPU features: detected: Data cache clean to Point of Deep Persistence607worker2 # [ 0.004876] CPU features: detected: Data cache clean to Point of Deep Persistence608client # [ 0.004828] CPU features: detected: Instruction cache invalidation not required for I/D coherence609lb # [ 0.004784] CPU features: detected: Data cache clean to Point of Persistence610worker2 # [ 0.004879] CPU features: detected: Data cache clean to Point of Persistence611worker1 # [ 0.161985] iommu: Default domain type: Translated612client # [ 0.004832] CPU features: detected: Data cache clean to the PoU not required for I/D coherence613lb # [ 0.004787] CPU features: detected: Data independent timing control (DIT)614worker2 # [ 0.004883] CPU features: detected: Data independent timing control (DIT)615worker1 # [ 0.161994] iommu: DMA domain TLB invalidation policy: strict mode616lb # [ 0.004790] CPU features: detected: E0PD617worker2 # [ 0.004886] CPU features: detected: E0PD618client # [ 0.004835] CPU features: detected: Common not Private translations619worker1 # [ 0.162352] SCSI subsystem initialized620lb # [ 0.004792] CPU features: detected: Enhanced Counter Virtualization621client # [ 0.004839] CPU features: detected: CRC32 instructions622worker2 # [ 0.004888] CPU features: detected: Enhanced Counter Virtualization623worker1 # [ 0.164299] usbcore: registered new interface driver usbfs624lb # [ 0.004795] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)625client # [ 0.004842] CPU features: detected: Data cache clean to Point of Deep Persistence626worker2 # [ 0.004891] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)627worker1 # [ 0.164330] usbcore: registered new interface driver hub628lb # [ 0.004799] CPU features: detected: Enhanced Virtualization Traps629worker2 # [ 0.004895] CPU features: detected: Enhanced Virtualization Traps630client # [ 0.004845] CPU features: detected: Data cache clean to Point of Persistence631worker1 # [ 0.164346] usbcore: registered new device driver usb632lb # [ 0.004802] CPU features: detected: Fine Grained Traps633worker2 # [ 0.004898] CPU features: detected: Fine Grained Traps634client # [ 0.004848] CPU features: detected: Data independent timing control (DIT)635worker1 # [ 0.164598] pps_core: LinuxPPS API ver. 1 registered636client # [ 0.004852] CPU features: detected: E0PD637lb # [ 0.004806] CPU features: detected: Generic authentication (architected QARMA5 algorithm)638worker2 # [ 0.004902] CPU features: detected: Generic authentication (architected QARMA5 algorithm)639worker1 # [ 0.164604] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>640lb # [ 0.004811] CPU features: detected: RCpc load-acquire (LDAPR)641client # [ 0.004854] CPU features: detected: Enhanced Counter Virtualization642worker2 # [ 0.004906] CPU features: detected: RCpc load-acquire (LDAPR)643worker1 # [ 0.164614] PTP clock support registered644lb # [ 0.004813] CPU features: detected: LSE atomic instructions645worker1 # [ 0.164661] EDAC MC: Ver: 3.0.0646client # [ 0.004857] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)647worker2 # [ 0.004909] CPU features: detected: LSE atomic instructions648lb # [ 0.004816] CPU features: detected: Privileged Access Never649worker2 # [ 0.004912] CPU features: detected: Privileged Access Never650client # [ 0.004861] CPU features: detected: Enhanced Virtualization Traps651worker1 # [ 0.169205] scmi_core: SCMI protocol bus registered652lb # [ 0.004819] CPU features: detected: PMUv3653worker2 # [ 0.004914] CPU features: detected: PMUv3654client # [ 0.004864] CPU features: detected: Fine Grained Traps655lb # [ 0.004822] CPU features: detected: RAS Extension Support656worker2 # [ 0.004917] CPU features: detected: RAS Extension Support657worker1 # [ 0.170164] FPGA manager framework658lb # [ 0.004825] CPU features: detected: RASv1p1 Extension Support659client # [ 0.004868] CPU features: detected: Generic authentication (architected QARMA5 algorithm)660worker1 # [ 0.171101] vgaarb: loaded661worker2 # [ 0.004920] CPU features: detected: RASv1p1 Extension Support662lb # [ 0.004827] CPU features: detected: Random Number Generator663client # [ 0.004873] CPU features: detected: RCpc load-acquire (LDAPR)664worker2 # [ 0.004923] CPU features: detected: Random Number Generator665worker1 # [ 0.171704] clocksource: Switched to clocksource arch_sys_counter666lb # [ 0.004830] CPU features: detected: Speculation barrier (SB)667client # [ 0.004876] CPU features: detected: LSE atomic instructions668worker2 # [ 0.004925] CPU features: detected: Speculation barrier (SB)669lb # [ 0.004833] CPU features: detected: Stage-2 Force Write-Back670client # [ 0.004879] CPU features: detected: Privileged Access Never671worker2 # [ 0.004928] CPU features: detected: Stage-2 Force Write-Back672client # [ 0.004881] CPU features: detected: PMUv3673lb # [ 0.004836] CPU features: detected: TLB range maintenance instructions674worker2 # [ 0.004931] CPU features: detected: TLB range maintenance instructions675client # [ 0.004884] CPU features: detected: RAS Extension Support676lb # [ 0.004840] CPU features: detected: Speculative Store Bypassing Safe (SSBS)677worker2 # [ 0.004936] CPU features: detected: Speculative Store Bypassing Safe (SSBS)678client # [ 0.004887] CPU features: detected: RASv1p1 Extension Support679lb # [ 0.004874] alternatives: applying system-wide alternatives680worker2 # [ 0.004971] alternatives: applying system-wide alternatives681client # [ 0.004890] CPU features: detected: Random Number Generator682worker1 # [ 0.175610] VFS: Disk quotas dquot_6.6.0683lb # [ 0.007688] CPU features: detected: BBM Level 2 without TLB conflict abort684worker2 # [ 0.007832] CPU features: detected: BBM Level 2 without TLB conflict abort685client # [ 0.004892] CPU features: detected: Speculation barrier (SB)686worker1 # [ 0.175646] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)687client # [ 0.004895] CPU features: detected: Stage-2 Force Write-Back688client # [ 0.004898] CPU features: detected: TLB range maintenance instructions689lb # [ 0.007866] Memory: 1405664K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)690worker2 # [ 0.008068] Memory: 1405708K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)691lb # [ 0.008251] devtmpfs: initialized692worker2 # [ 0.008432] devtmpfs: initialized693client # [ 0.004903] CPU features: detected: Speculative Store Bypassing Safe (SSBS)694lb # [ 0.010013] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)695client # [ 0.004938] alternatives: applying system-wide alternatives696worker2 # [ 0.010209] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)697worker1 # [ 0.179186] netfs: FS-Cache loaded698client # [ 0.007804] CPU features: detected: BBM Level 2 without TLB conflict abort699worker1 # [ 0.179319] pnp: PnP ACPI: disabled700lb # [ 0.010040] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).701worker2 # [ 0.010237] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).702lb # [ 0.010201] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL703worker2 # [ 0.010406] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL704lb # [ 0.010205] 0 pages in range for non-PLT usage705worker2 # [ 0.010410] 0 pages in range for non-PLT usage706client # [ 0.007994] Memory: 1405620K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)707lb # [ 0.010206] 508272 pages in range for PLT usage708client # [ 0.008392] devtmpfs: initialized709worker2 # [ 0.010411] 508272 pages in range for PLT usage710lb # [ 0.010308] pinctrl core: initialized pinctrl subsystem711worker2 # [ 0.010518] pinctrl core: initialized pinctrl subsystem712lb # [ 0.011016] DMI not present or invalid.713client # [ 0.010203] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)714worker2 # [ 0.011290] DMI not present or invalid.715lb # [ 0.014240] NET: Registered PF_NETLINK/PF_ROUTE protocol family716worker2 # [ 0.014511] NET: Registered PF_NETLINK/PF_ROUTE protocol family717client # [ 0.010225] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).718worker1 # [ 0.183118] NET: Registered PF_INET protocol family719lb # [ 0.016406] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations720worker2 # [ 0.016668] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations721client # [ 0.010388] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL722worker1 # [ 0.183374] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)723client # [ 0.010392] 0 pages in range for non-PLT usage724lb # [ 0.016586] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations725worker2 # [ 0.016850] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations726client # [ 0.010393] 508272 pages in range for PLT usage727lb # [ 0.016782] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations728client # [ 0.010501] pinctrl core: initialized pinctrl subsystem729worker2 # [ 0.017047] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations730client # [ 0.011274] DMI not present or invalid.731lb # [ 0.016802] audit: initializing netlink subsys (disabled)732worker2 # [ 0.017070] audit: initializing netlink subsys (disabled)733client # [ 0.014376] NET: Registered PF_NETLINK/PF_ROUTE protocol family734lb # [ 0.017326] thermal_sys: Registered thermal governor 'fair_share'735worker2 # [ 0.017629] thermal_sys: Registered thermal governor 'fair_share'736lb # [ 0.017328] thermal_sys: Registered thermal governor 'bang_bang'737client # [ 0.016533] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations738worker2 # [ 0.017632] thermal_sys: Registered thermal governor 'bang_bang'739lb # [ 0.017332] thermal_sys: Registered thermal governor 'step_wise'740client # [ 0.016721] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations741worker2 # [ 0.017636] thermal_sys: Registered thermal governor 'step_wise'742lb # [ 0.017334] thermal_sys: Registered thermal governor 'user_space'743worker2 # [ 0.017638] thermal_sys: Registered thermal governor 'user_space'744client # [ 0.016916] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations745lb # [ 0.017339] thermal_sys: Registered thermal governor 'power_allocator'746worker2 # [ 0.017643] thermal_sys: Registered thermal governor 'power_allocator'747client # [ 0.016937] audit: initializing netlink subsys (disabled)748lb # [ 0.017363] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1749client # [ 0.017510] thermal_sys: Registered thermal governor 'fair_share'750worker2 # [ 0.017667] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1751lb # [ 0.017373] cpuidle: using governor ladder752worker2 # [ 0.017677] cpuidle: using governor ladder753client # [ 0.017512] thermal_sys: Registered thermal governor 'bang_bang'754lb # [ 0.017379] cpuidle: using governor menu755worker2 # [ 0.017682] cpuidle: using governor menu756client # [ 0.017516] thermal_sys: Registered thermal governor 'step_wise'757lb # [ 0.017576] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.758worker2 # [ 0.017887] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.759client # [ 0.017518] thermal_sys: Registered thermal governor 'user_space'760lb # [ 0.017591] ASID allocator initialised with 65536 entries761worker2 # [ 0.017903] ASID allocator initialised with 65536 entries762lb # [ 0.018782] Serial: AMBA PL011 UART driver763client # [ 0.017523] thermal_sys: Registered thermal governor 'power_allocator'764worker2 # [ 0.019034] Serial: AMBA PL011 UART driver765client # [ 0.017545] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1766worker2 # [ 0.024156] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1767client # [ 0.017555] cpuidle: using governor ladder768lb # [ 0.024020] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1769worker2 # [ 0.024293] printk: console [ttyAMA0] enabled770client # [ 0.017561] cpuidle: using governor menu771lb # [ 0.024185] printk: console [ttyAMA0] enabled772client # [ 0.017736] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.773client # [ 0.017751] ASID allocator initialised with 65536 entries774client # [ 0.018906] Serial: AMBA PL011 UART driver775client # [ 0.024084] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1776client # [ 0.024217] printk: console [ttyAMA0] enabled777lb # [ 0.140990] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages778worker2 # [ 0.141686] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages779lb # [ 0.141006] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page780worker2 # [ 0.141703] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page781lb # [ 0.141011] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages782worker2 # [ 0.141708] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages783lb # [ 0.141015] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page784worker2 # [ 0.141713] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page785lb # [ 0.141020] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages786client # [ 0.141151] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages787worker2 # [ 0.141717] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages788lb # [ 0.141024] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page789client # [ 0.141166] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page790worker2 # [ 0.141722] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page791lb # [ 0.141028] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages792client # [ 0.141171] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages793worker2 # [ 0.141726] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages794lb # [ 0.141033] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page795client # [ 0.141176] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page796worker2 # [ 0.141730] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page797client # [ 0.141180] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages798client # [ 0.141185] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page799lb # [ 0.148407] fbcon: Taking over console800worker2 # [ 0.149124] fbcon: Taking over console801client # [ 0.141189] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages802lb # [ 0.148421] ACPI: Interpreter disabled.803worker2 # [ 0.149138] ACPI: Interpreter disabled.804client # [ 0.141193] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page805lb # [ 0.150305] iommu: Default domain type: Translated806worker2 # [ 0.150959] iommu: Default domain type: Translated807lb # [ 0.150316] iommu: DMA domain TLB invalidation policy: strict mode808worker2 # [ 0.150969] iommu: DMA domain TLB invalidation policy: strict mode809lb # [ 0.151978] SCSI subsystem initialized810worker2 # [ 0.152648] SCSI subsystem initialized811client # [ 0.148494] fbcon: Taking over console812client # [ 0.148508] ACPI: Interpreter disabled.813lb # [ 0.152663] usbcore: registered new interface driver usbfs814worker2 # [ 0.153324] usbcore: registered new interface driver usbfs815lb # [ 0.152695] usbcore: registered new interface driver hub816worker2 # [ 0.153355] usbcore: registered new interface driver hub817lb # [ 0.152718] usbcore: registered new device driver usb818worker2 # [ 0.153377] usbcore: registered new device driver usb819lb # [ 0.152985] pps_core: LinuxPPS API ver. 1 registered820worker2 # [ 0.153633] pps_core: LinuxPPS API ver. 1 registered821lb # [ 0.152991] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>822worker2 # [ 0.153640] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>823lb # [ 0.153002] PTP clock support registered824worker2 # [ 0.153649] PTP clock support registered825lb # [ 0.153057] EDAC MC: Ver: 3.0.0826worker2 # [ 0.153697] EDAC MC: Ver: 3.0.0827worker1 # [ 0.212236] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)828worker1 # [ 0.212272] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)829worker1 # [ 0.212299] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)830client # [ 0.157494] iommu: Default domain type: Translated831worker1 # [ 0.212377] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)832client # [ 0.157504] iommu: DMA domain TLB invalidation policy: strict mode833worker1 # [ 0.212520] TCP: Hash tables configured (established 16384 bind 16384)834client # [ 0.157816] SCSI subsystem initialized835worker1 # [ 0.212589] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)836client # [ 0.159796] usbcore: registered new interface driver usbfs837worker1 # [ 0.212636] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)838client # [ 0.159827] usbcore: registered new interface driver hub839worker1 # [ 0.212672] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)840client # [ 0.159850] usbcore: registered new device driver usb841worker1 # [ 0.212768] NET: Registered PF_UNIX/PF_LOCAL protocol family842client # [ 0.160122] pps_core: LinuxPPS API ver. 1 registered843worker1 # [ 0.212787] NET: Registered PF_XDP protocol family844lb # [ 0.165950] scmi_core: SCMI protocol bus registered845worker1 # [ 0.212802] PCI: CLS 0 bytes, default 64846worker2 # [ 0.166621] scmi_core: SCMI protocol bus registered847client # [ 0.160128] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>848client # [ 0.160139] PTP clock support registered849worker1 # [ 0.213016] Trying to unpack rootfs image as initramfs...850lb # [ 0.166918] FPGA manager framework851client # [ 0.160189] EDAC MC: Ver: 3.0.0852worker2 # [ 0.167558] FPGA manager framework853lb # [ 0.167887] vgaarb: loaded854worker2 # [ 0.168499] vgaarb: loaded855client # [ 0.164781] scmi_core: SCMI protocol bus registered856lb # [ 0.168493] clocksource: Switched to clocksource arch_sys_counter857worker2 # [ 0.169112] clocksource: Switched to clocksource arch_sys_counter858client # [ 0.165751] FPGA manager framework859client # [ 0.166670] vgaarb: loaded860client # [ 0.167272] clocksource: Switched to clocksource arch_sys_counter861lb # [ 0.170807] VFS: Disk quotas dquot_6.6.0862worker2 # [ 0.171369] VFS: Disk quotas dquot_6.6.0863lb # [ 0.170841] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)864worker2 # [ 0.171405] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)865worker1 # [ 0.229474] kvm [1]: HYP mode not available866lb # [ 0.174395] netfs: FS-Cache loaded867worker2 # [ 0.174952] netfs: FS-Cache loaded868lb # [ 0.174506] pnp: PnP ACPI: disabled869worker2 # [ 0.175070] pnp: PnP ACPI: disabled870client # [ 0.173289] VFS: Disk quotas dquot_6.6.0871client # [ 0.173330] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)872client # [ 0.174945] netfs: FS-Cache loaded873client # [ 0.175060] pnp: PnP ACPI: disabled874lb # [ 0.178433] NET: Registered PF_INET protocol family875worker2 # [ 0.178881] NET: Registered PF_INET protocol family876lb # [ 0.178689] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)877worker2 # [ 0.179127] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)878client # [ 0.178915] NET: Registered PF_INET protocol family879client # [ 0.179162] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)880lb # [ 0.206793] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881worker2 # [ 0.207570] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)882lb # [ 0.206834] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)883worker2 # [ 0.207607] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)884lb # [ 0.206861] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)885worker2 # [ 0.207634] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)886lb # [ 0.206931] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)887worker2 # [ 0.207708] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)888lb # [ 0.207075] TCP: Hash tables configured (established 16384 bind 16384)889worker2 # [ 0.207851] TCP: Hash tables configured (established 16384 bind 16384)890lb # [ 0.207156] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)891worker2 # [ 0.207923] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)892lb # [ 0.207189] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)893worker2 # [ 0.207962] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)894client # [ 0.208134] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)895lb # [ 0.207224] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)896worker2 # [ 0.207998] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)897lb # [ 0.207328] NET: Registered PF_UNIX/PF_LOCAL protocol family898client # [ 0.208174] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)899worker2 # [ 0.208096] NET: Registered PF_UNIX/PF_LOCAL protocol family900lb # [ 0.207352] NET: Registered PF_XDP protocol family901worker2 # [ 0.208146] NET: Registered PF_XDP protocol family902lb # [ 0.207368] PCI: CLS 0 bytes, default 64903client # [ 0.208197] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)904worker2 # [ 0.208162] PCI: CLS 0 bytes, default 64905lb # [ 0.207611] Trying to unpack rootfs image as initramfs...906client # [ 0.208265] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)907worker2 # [ 0.208391] Trying to unpack rootfs image as initramfs...908client # [ 0.208416] TCP: Hash tables configured (established 16384 bind 16384)909client # [ 0.208491] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)910client # [ 0.208525] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)911lb # [ 0.222097] kvm [1]: HYP mode not available912worker2 # [ 0.222785] kvm [1]: HYP mode not available913client # [ 0.208561] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914client # [ 0.208654] NET: Registered PF_UNIX/PF_LOCAL protocol family915client # [ 0.208705] NET: Registered PF_XDP protocol family916client # [ 0.208723] PCI: CLS 0 bytes, default 64917client # [ 0.208955] Trying to unpack rootfs image as initramfs...918client # [ 0.224803] kvm [1]: HYP mode not available919worker1 # [ 0.320224] Initialise system trusted keyrings920worker1 # [ 0.320952] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker1 # [ 0.322189] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker1 # [ 0.322950] 9p: Installing v9fs 9p2000 file system support923worker1 # [ 0.343528] Key type asymmetric registered924worker1 # [ 0.343549] Asymmetric key parser 'x509' registered925worker1 # [ 0.343611] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)926worker1 # [ 0.351773] io scheduler mq-deadline registered927worker1 # [ 0.351790] io scheduler kyber registered928worker1 # [ 0.356813] pl061_gpio 9030000.pl061: PL061 GPIO chip registered929worker1 # [ 0.363730] ledtrig-cpu: registered to indicate activity on CPUs930worker1 # [ 0.364149] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:931worker1 # [ 0.364169] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000932worker1 # [ 0.364181] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000933worker1 # [ 0.364189] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000934lb # [ 0.315008] Initialise system trusted keyrings935worker1 # [ 0.364211] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits936worker2 # [ 0.316369] Initialise system trusted keyrings937lb # [ 0.315757] workingset: timestamp_bits=42 max_order=19 bucket_order=0938worker1 # [ 0.364236] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]939worker2 # [ 0.317095] workingset: timestamp_bits=42 max_order=19 bucket_order=0940lb # [ 0.317054] squashfs: version 4.0 (2009/01/31) Phillip Lougher941worker1 # [ 0.364323] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00942worker1 # [ 0.364333] pci_bus 0000:00: root bus resource [bus 00-ff]943worker2 # [ 0.318311] squashfs: version 4.0 (2009/01/31) Phillip Lougher944worker1 # [ 0.364339] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]945client # [ 0.315917] Initialise system trusted keyrings946worker1 # [ 0.364344] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]947client # [ 0.316654] workingset: timestamp_bits=42 max_order=19 bucket_order=0948worker1 # [ 0.364350] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]949lb # [ 0.317803] 9p: Installing v9fs 9p2000 file system support950worker2 # [ 0.319077] 9p: Installing v9fs 9p2000 file system support951client # [ 0.317902] squashfs: version 4.0 (2009/01/31) Phillip Lougher952worker1 # [ 0.364414] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint953worker1 # [ 0.364845] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint954client # [ 0.318678] 9p: Installing v9fs 9p2000 file system support955worker1 # [ 0.365028] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]956worker1 # [ 0.365044] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]957worker1 # [ 0.365074] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]958worker1 # [ 0.365091] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]959worker1 # [ 0.365559] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint960worker1 # [ 0.365738] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]961worker1 # [ 0.365754] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]962worker1 # [ 0.365783] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]963worker1 # [ 0.366224] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint964worker1 # [ 0.366402] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]965worker1 # [ 0.366418] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]966worker1 # [ 0.366448] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]967worker1 # [ 0.366887] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint968worker1 # [ 0.367066] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]969worker1 # [ 0.367082] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]970worker1 # [ 0.367111] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]971worker1 # [ 0.367129] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]972worker1 # [ 0.367571] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint973worker1 # [ 0.367776] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]974worker1 # [ 0.367805] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975worker1 # [ 0.368249] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint976worker1 # [ 0.368432] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]977worker1 # [ 0.368461] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]978lb # [ 0.341776] Key type asymmetric registered979worker1 # [ 0.368849] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint980lb # [ 0.341796] Asymmetric key parser 'x509' registered981worker1 # [ 0.369027] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]982worker2 # [ 0.343047] Key type asymmetric registered983worker2 # [ 0.343067] Asymmetric key parser 'x509' registered984lb # [ 0.341859] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)985worker1 # [ 0.369287] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint986lb # [ 0.343946] io scheduler mq-deadline registered987worker2 # [ 0.343122] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)988worker1 # [ 0.369470] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]989lb # [ 0.343957] io scheduler kyber registered990worker1 # [ 0.369499] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]991worker1 # [ 0.369946] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint992worker1 # [ 0.370127] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]993worker1 # [ 0.370156] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker1 # [ 0.370596] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint995worker1 # [ 0.370778] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]996worker1 # [ 0.370807] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]997client # [ 0.347327] Key type asymmetric registered998worker1 # [ 0.371251] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint999client # [ 0.347345] Asymmetric key parser 'x509' registered1000worker1 # [ 0.371512] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1001client # [ 0.347415] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1002worker2 # [ 0.349181] io scheduler mq-deadline registered1003worker1 # [ 0.371530] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1004client # [ 0.349532] io scheduler mq-deadline registered1005worker2 # [ 0.349196] io scheduler kyber registered1006client # [ 0.349542] io scheduler kyber registered1007worker1 # [ 0.371559] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1008lb # [ 0.357286] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1009worker1 # [ 0.420088] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1010worker1 # [ 0.420285] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1011client # [ 0.359390] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1012worker1 # [ 0.420302] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1013lb # [ 0.358583] ledtrig-cpu: registered to indicate activity on CPUs1014worker1 # [ 0.420332] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1015worker2 # [ 0.357265] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1016worker1 # [ 0.420905] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1017lb # [ 0.358962] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1018worker2 # [ 0.357847] ledtrig-cpu: registered to indicate activity on CPUs1019worker1 # [ 0.420916] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1020worker2 # [ 0.358213] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1021lb # [ 0.358978] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001022client # [ 0.360867] ledtrig-cpu: registered to indicate activity on CPUs1023worker1 # [ 0.420922] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1024worker2 # [ 0.358231] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001025client # [ 0.361242] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1026lb # [ 0.358990] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001027worker1 # [ 0.420968] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1028worker2 # [ 0.358243] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001029lb # [ 0.358999] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001030client # [ 0.361260] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001031worker1 # [ 0.421015] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1032lb # [ 0.359018] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1033worker2 # [ 0.358251] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001034client # [ 0.361273] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001035worker1 # [ 0.421063] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1036worker2 # [ 0.358270] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1037lb # [ 0.359039] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1038client # [ 0.361281] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001039worker1 # [ 0.421110] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1040lb # [ 0.359110] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001041worker2 # [ 0.358293] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1042client # [ 0.361302] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1043lb # [ 0.359119] pci_bus 0000:00: root bus resource [bus 00-ff]1044worker1 # [ 0.421157] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1045worker2 # [ 0.358363] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001046client # [ 0.361327] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1047lb # [ 0.359126] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1048worker1 # [ 0.421205] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1049worker2 # [ 0.358373] pci_bus 0000:00: root bus resource [bus 00-ff]1050lb # [ 0.359132] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1051client # [ 0.361414] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001052worker2 # [ 0.358378] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1053worker1 # [ 0.421261] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1054client # [ 0.361424] pci_bus 0000:00: root bus resource [bus 00-ff]1055lb # [ 0.359137] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1056worker2 # [ 0.358384] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1057worker1 # [ 0.421308] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1058client # [ 0.361430] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1059worker2 # [ 0.358389] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1060lb # [ 0.359190] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1061client # [ 0.361436] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1062worker1 # [ 0.421356] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1063worker2 # [ 0.358460] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1064lb # [ 0.359637] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1065client # [ 0.361441] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1066worker1 # [ 0.421468] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1067lb # [ 0.359822] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1068worker2 # [ 0.358905] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1069worker1 # [ 0.421521] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1070client # [ 0.361496] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1071lb # [ 0.359838] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1072worker2 # [ 0.359092] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1073lb # [ 0.359868] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074worker1 # [ 0.421543] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1075client # [ 0.361947] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1076worker2 # [ 0.359109] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1077lb # [ 0.359885] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1078client # [ 0.362137] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1079worker1 # [ 0.421565] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1080worker2 # [ 0.359138] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081client # [ 0.362154] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1082worker1 # [ 0.421587] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1083worker2 # [ 0.359154] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1084lb # [ 0.360329] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1085client # [ 0.362184] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086lb # [ 0.360529] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1087worker1 # [ 0.421609] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1088worker2 # [ 0.359604] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1089client # [ 0.362200] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1090lb # [ 0.360544] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1091worker1 # [ 0.421630] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1092worker2 # [ 0.359789] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1093lb # [ 0.360574] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094client # [ 0.362665] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1095worker1 # [ 0.421652] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1096worker2 # [ 0.359805] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1097client # [ 0.362853] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1098lb # [ 0.361028] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1099worker1 # [ 0.421678] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1100worker2 # [ 0.359835] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101client # [ 0.362869] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1102lb # [ 0.361210] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1103worker1 # [ 0.421700] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1104worker2 # [ 0.360293] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1105lb # [ 0.361226] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1106client # [ 0.362899] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1107worker1 # [ 0.421723] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1108worker2 # [ 0.360477] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1109lb # [ 0.361255] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1110worker2 # [ 0.360493] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1111worker1 # [ 0.421746] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1112lb # [ 0.361726] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1113worker2 # [ 0.360522] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114worker1 # [ 0.421768] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1115lb # [ 0.361910] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1116client # [ 0.381763] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1117worker1 # [ 0.421789] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1118worker2 # [ 0.360981] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1119lb # [ 0.361926] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1120client # [ 0.381955] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1121worker1 # [ 0.421810] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1122worker2 # [ 0.361188] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1123lb # [ 0.361955] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124client # [ 0.381971] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1125worker1 # [ 0.421832] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1126worker2 # [ 0.361204] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1127lb # [ 0.361974] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1128client # [ 0.382002] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129worker1 # [ 0.421853] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1130worker2 # [ 0.361233] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1131lb # [ 0.362440] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1132worker2 # [ 0.361251] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1133worker1 # [ 0.421874] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1134client # [ 0.382469] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1135lb # [ 0.362627] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1136client # [ 0.382657] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1137worker1 # [ 0.421895] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1138worker2 # [ 0.361704] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1139lb # [ 0.362657] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1140worker1 # [ 0.421921] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1141client # [ 0.382673] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1142worker2 # [ 0.361893] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker1 # [ 0.421930] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1144client # [ 0.382703] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145lb # [ 0.363104] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1146worker2 # [ 0.361926] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147worker1 # [ 0.421935] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1148client # [ 0.382722] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1149lb # [ 0.363290] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1150worker2 # [ 0.362393] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1151worker1 # [ 0.422743] pci 0000:00:07.0: enabling device (0000 -> 0002)1152lb # [ 0.363320] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153client # [ 0.383187] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1154worker2 # [ 0.362584] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1155lb # [ 0.363855] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1156worker2 # [ 0.362614] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1157lb # [ 0.364038] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1158worker2 # [ 0.363007] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1159lb # [ 0.364284] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1160worker2 # [ 0.363189] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1161lb # [ 0.364472] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1162worker2 # [ 0.363439] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1163worker2 # [ 0.363627] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1164worker2 # [ 0.363656] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1165worker2 # [ 0.364113] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1166worker2 # [ 0.364301] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1167worker2 # [ 0.364331] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1168worker2 # [ 0.364780] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1169client # [ 0.395440] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1170worker2 # [ 0.364969] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1171client # [ 0.395482] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1172worker2 # [ 0.364999] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1173client # [ 0.395974] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1174client # [ 0.396171] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1175client # [ 0.396201] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176client # [ 0.396653] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1177client # [ 0.396840] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1178lb # [ 0.404585] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1179client # [ 0.397103] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180client # [ 0.397295] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1181lb # [ 0.405081] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1182client # [ 0.397325] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183lb # [ 0.405278] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1184lb # [ 0.405308] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1185client # [ 0.397788] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1186client # [ 0.397980] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1187lb # [ 0.405757] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1188client # [ 0.398011] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1189lb # [ 0.405943] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1190lb # [ 0.405973] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1191client # [ 0.398468] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1192client # [ 0.398658] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1193lb # [ 0.406430] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1194client # [ 0.398688] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1195lb # [ 0.406722] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1196lb # [ 0.406739] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1197client # [ 0.399152] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1198lb # [ 0.406769] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1199worker2 # [ 0.413572] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1200worker2 # [ 0.413844] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1201lb # [ 0.407221] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1202worker1 # [ 0.478887] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1203worker2 # [ 0.413862] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1204lb # [ 0.407402] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1205worker2 # [ 0.413892] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1206lb # [ 0.407418] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1207lb # [ 0.407448] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1208worker2 # [ 0.414362] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1209worker2 # [ 0.414548] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1210lb # [ 0.408015] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1211worker2 # [ 0.414564] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1212lb # [ 0.408027] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1213worker2 # [ 0.414594] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1214lb # [ 0.408033] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1215worker2 # [ 0.415151] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1216lb # [ 0.408078] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1217worker2 # [ 0.415162] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1218lb # [ 0.408124] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1219worker2 # [ 0.415168] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1220lb # [ 0.408171] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1221worker2 # [ 0.415212] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1222client # [ 0.419482] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1223lb # [ 0.408219] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1224client # [ 0.419508] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1225worker2 # [ 0.415259] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1226lb # [ 0.408266] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1227client # [ 0.419538] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1228worker2 # [ 0.415306] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1229lb # [ 0.408312] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1230client # [ 0.420034] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1231worker2 # [ 0.415354] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1232client # [ 0.420226] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1233lb # [ 0.408358] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1234worker2 # [ 0.415401] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1235client # [ 0.420243] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1236worker1 # [ 0.489034] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1237lb # [ 0.408404] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1238worker2 # [ 0.415449] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1239client # [ 0.420282] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1240lb # [ 0.408449] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1241client # [ 0.420859] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1242worker2 # [ 0.415497] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1243client # [ 0.420871] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1244worker2 # [ 0.415544] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1245client # [ 0.420876] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1246worker2 # [ 0.415591] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1247client # [ 0.420921] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1248worker2 # [ 0.415659] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1249worker2 # [ 0.415704] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1250client # [ 0.420969] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1251worker2 # [ 0.415726] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1252client # [ 0.421016] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1253worker2 # [ 0.415747] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1254client # [ 0.421064] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1255worker2 # [ 0.415769] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1256client # [ 0.421112] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1257worker2 # [ 0.415791] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1258client # [ 0.421159] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1259worker2 # [ 0.415812] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1260lb # [ 0.436573] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1261client # [ 0.421207] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1262worker2 # [ 0.415834] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1263lb # [ 0.436645] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1264worker2 # [ 0.415860] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1265lb # [ 0.436668] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1266client # [ 0.421254] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1267lb # [ 0.436690] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1268worker2 # [ 0.415883] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1269client # [ 0.421301] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1270lb # [ 0.436711] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1271worker2 # [ 0.415905] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1272client # [ 0.421401] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1273lb # [ 0.436732] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1274worker2 # [ 0.415927] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1275client # [ 0.421454] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1276lb # [ 0.436754] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1277worker2 # [ 0.415956] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1278client # [ 0.421476] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1279worker2 # [ 0.415978] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1280lb # [ 0.436776] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1281worker1 # [ 0.500274] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1282client # [ 0.421498] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1283worker2 # [ 0.415999] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1284lb # [ 0.436801] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1285worker1 # [ 0.502163] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1286client # [ 0.421519] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1287worker2 # [ 0.416020] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1288lb # [ 0.436823] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1289worker1 # [ 0.504110] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1290worker2 # [ 0.416041] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1291client # [ 0.421541] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1292lb # [ 0.436845] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1293worker1 # [ 0.506364] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1294worker2 # [ 0.416063] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1295client # [ 0.421562] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1296lb # [ 0.436867] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1297worker2 # [ 0.416084] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1298client # [ 0.421584] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1299lb # [ 0.436889] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1300worker2 # [ 0.416109] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1301client # [ 0.421609] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1302lb # [ 0.436910] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1303worker2 # [ 0.416119] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1304client # [ 0.421631] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1305lb # [ 0.436931] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1306worker2 # [ 0.416123] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1307lb # [ 0.436951] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1308client # [ 0.421653] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1309worker2 # [ 0.416930] pci 0000:00:07.0: enabling device (0000 -> 0002)1310lb # [ 0.436972] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1311client # [ 0.421676] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1312lb # [ 0.436992] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1313client # [ 0.421697] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1314lb # [ 0.437013] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1315client # [ 0.421719] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1316lb # [ 0.437039] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1317client # [ 0.421740] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1318lb # [ 0.437049] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1319client # [ 0.421762] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1320lb # [ 0.437053] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1321client # [ 0.421782] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1322lb # [ 0.437861] pci 0000:00:07.0: enabling device (0000 -> 0002)1323client # [ 0.421803] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1324client # [ 0.421825] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1325worker1 # [ 0.516207] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1326client # [ 0.421850] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1327worker1 # [ 0.517965] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1328client # [ 0.421859] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1329worker1 # [ 0.519698] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1330client # [ 0.421864] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1331client # [ 0.422677] pci 0000:00:07.0: enabling device (0000 -> 0002)1332worker2 # [ 0.461230] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 43266 usecs1333worker1 # [ 0.523830] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1334worker1 # [ 0.526093] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1335lb # [ 0.476288] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1336worker2 # [ 0.481654] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1337worker2 # [ 0.483661] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1338worker1 # [ 0.547645] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1339client # [ 0.481952] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1340lb # [ 0.485794] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1341worker1 # [ 0.550153] msm_serial: driver initialized1342lb # [ 0.488079] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1343worker1 # [ 0.550304] SuperH (H)SCI(F) driver initialized1344worker1 # [ 0.550357] STM32 USART driver initialized1345client # [ 0.487901] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1346client # [ 0.489893] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1347worker2 # [ 0.493690] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1348lb # [ 0.497729] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1349worker2 # [ 0.496048] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1350worker2 # [ 0.498104] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1351worker2 # [ 0.500336] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1352lb # [ 0.502189] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1353lb # [ 0.504339] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1354client # [ 0.499850] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1355client # [ 0.501872] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1356worker2 # [ 0.510388] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1357worker2 # [ 0.512244] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1358client # [ 0.512072] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1359client # [ 0.513982] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1360lb # [ 0.517863] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1361lb # [ 0.519735] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1362worker1 # [ 0.584283] loop: module loaded1363worker1 # [ 0.584470] virtio_blk virtio2: 1/0/0 default/read/poll queues1364lb # [ 0.523180] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1365worker2 # [ 0.522651] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1366worker1 # [ 0.585341] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1367worker2 # [ 0.524649] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1368client # [ 0.517385] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1369client # [ 0.519230] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1370client # [ 0.521171] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1371worker2 # [ 0.528602] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1372lb # [ 0.525932] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1373lb # [ 0.528232] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1374worker1 # [ 0.592270] megasas: 07.734.00.00-rc11375worker1 # [ 0.592972] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1376worker1 # [ 0.595134] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001377worker1 # [ 0.595158] Intel/Sharp Extended Query Table at 0x00311378worker1 # [ 0.596703] Using buffer write method1379worker1 # [ 0.596769] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1380worker1 # [ 0.598478] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001381worker1 # [ 0.598498] Intel/Sharp Extended Query Table at 0x00311382lb # [ 0.545702] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1383worker2 # [ 0.546345] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1384client # [ 0.540374] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1385lb # [ 0.548229] msm_serial: driver initialized1386lb # [ 0.548380] SuperH (H)SCI(F) driver initialized1387lb # [ 0.548436] STM32 USART driver initialized1388client # [ 0.546029] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1389client # [ 0.548522] msm_serial: driver initialized1390worker1 # [ 0.615764] Using buffer write method1391client # [ 0.548665] SuperH (H)SCI(F) driver initialized1392worker1 # [ 0.615790] Concatenating MTD devices:1393client # [ 0.548719] STM32 USART driver initialized1394worker1 # [ 0.615794] (0): "0.flash"1395worker1 # [ 0.615798] (1): "0.flash"1396worker2 # [ 0.554186] msm_serial: driver initialized1397worker1 # [ 0.615806] into device "0.flash"1398worker2 # [ 0.554323] SuperH (H)SCI(F) driver initialized1399worker2 # [ 0.554377] STM32 USART driver initialized1400lb # [ 0.581987] loop: module loaded1401lb # [ 0.582153] virtio_blk virtio2: 1/0/0 default/read/poll queues1402worker2 # [ 0.583850] loop: module loaded1403lb # [ 0.582869] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1404worker2 # [ 0.584032] virtio_blk virtio2: 1/0/0 default/read/poll queues1405worker2 # [ 0.584753] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1406client # [ 0.583864] loop: module loaded1407client # [ 0.584037] virtio_blk virtio2: 1/0/0 default/read/poll queues1408client # [ 0.584827] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1409lb # [ 0.589090] megasas: 07.734.00.00-rc11410worker2 # [ 0.589668] megasas: 07.734.00.00-rc11411lb # [ 0.589800] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1412worker2 # [ 0.590335] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1413lb # [ 0.591921] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001414lb # [ 0.591944] Intel/Sharp Extended Query Table at 0x00311415client # [ 0.591885] megasas: 07.734.00.00-rc11416client # [ 0.592698] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1417client # [ 0.594815] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001418lb # [ 0.601538] Using buffer write method1419client # [ 0.594870] Intel/Sharp Extended Query Table at 0x00311420worker2 # [ 0.601154] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001421client # [ 0.596397] Using buffer write method1422lb # [ 0.601601] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1423worker2 # [ 0.601193] Intel/Sharp Extended Query Table at 0x00311424worker2 # [ 0.603922] Using buffer write method1425client # [ 0.596457] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1426lb # [ 0.604207] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001427worker2 # [ 0.603990] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1428lb # [ 0.604228] Intel/Sharp Extended Query Table at 0x00311429client # [ 0.598117] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001430client # [ 0.598140] Intel/Sharp Extended Query Table at 0x00311431lb # [ 0.613403] Using buffer write method1432lb # [ 0.613427] Concatenating MTD devices:1433lb # [ 0.613431] (0): "0.flash"1434client # [ 0.611771] Using buffer write method1435lb # [ 0.613435] (1): "0.flash"1436client # [ 0.611794] Concatenating MTD devices:1437lb # [ 0.613443] into device "0.flash"1438client # [ 0.611799] (0): "0.flash"1439client # [ 0.611803] (1): "0.flash"1440client # [ 0.611810] into device "0.flash"1441worker2 # [ 0.613752] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001442worker2 # [ 0.613775] Intel/Sharp Extended Query Table at 0x00311443worker2 # [ 0.616383] Using buffer write method1444worker2 # [ 0.616408] Concatenating MTD devices:1445worker2 # [ 0.616411] (0): "0.flash"1446worker2 # [ 0.616420] (1): "0.flash"1447worker2 # [ 0.616424] into device "0.flash"1448worker1 # [ 0.843499] Freeing initrd memory: 26976K1449worker1 # [ 0.849526] tun: Universal TUN/TAP device driver, 1.61450worker1 # [ 0.853173] thunder_xcv, ver 1.01451worker1 # [ 0.853214] thunder_bgx, ver 1.01452worker1 # [ 0.853245] nicpf, ver 1.01453worker1 # [ 0.853821] e1000: Intel(R) PRO/1000 Network Driver1454worker1 # [ 0.853828] e1000: Copyright (c) 1999-2006 Intel Corporation.1455worker1 # [ 0.853852] e1000e: Intel(R) PRO/1000 Network Driver1456worker1 # [ 0.853860] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457worker1 # [ 0.853889] igb: Intel(R) Gigabit Ethernet Network Driver1458worker1 # [ 0.853895] igb: Copyright (c) 2007-2014 Intel Corporation.1459worker1 # [ 0.853917] igbvf: Intel(R) Gigabit Virtual Function Network Driver1460worker1 # [ 0.853923] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1461worker1 # [ 0.854062] sky2: driver version 1.301462worker1 # [ 0.855616] usbcore: registered new interface driver usb-storage1463worker1 # [ 0.864131] ehci-pci 0000:00:07.0: EHCI Host Controller1464worker1 # [ 0.864158] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11465worker1 # [ 0.864411] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001466worker1 # [ 0.866756] usbcore: registered new interface driver usbserial_generic1467worker1 # [ 0.866774] usbserial: USB Serial support registered for generic1468worker1 # [ 0.867406] hv_vmbus: registering driver hyperv_keyboard1469worker1 # [ 0.870413] rtc-pl031 9010000.pl031: registered as rtc01470worker1 # [ 0.870440] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T10:34:14 UTC (1790159654)1471worker1 # [ 0.870736] i2c_dev: i2c /dev entries driver1472worker1 # [ 0.876092] sdhci: Secure Digital Host Controller Interface driver1473worker1 # [ 0.876102] sdhci: Copyright(c) Pierre Ossman1474worker1 # [ 0.876367] Synopsys Designware Multimedia Card Interface Driver1475worker1 # [ 0.876738] sdhci-pltfm: SDHCI platform and OF driver helper1476worker1 # [ 0.878291] hid: raw HID events driver (C) Jiri Kosina1477worker1 # [ 0.878526] usbcore: registered new interface driver usbhid1478worker1 # [ 0.878532] usbhid: USB HID core driver1479worker1 # [ 0.884093] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001480worker1 # [ 0.885122] hub 1-0:1.0: USB hub found1481worker1 # [ 0.885618] hub 1-0:1.0: 6 ports detected1482worker1 # [ 0.886584] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1483worker1 # [ 0.889218] drop_monitor: Initializing network drop monitor service1484worker1 # [ 0.889363] NET: Registered PF_INET6 protocol family1485worker1 # [ 0.891236] Segment Routing with IPv61486worker1 # [ 0.891270] In-situ OAM (IOAM) with IPv61487worker1 # [ 0.891298] NET: Registered PF_PACKET protocol family1488worker1 # [ 0.892949] 9pnet: Installing 9P2000 support1489worker1 # [ 0.892997] Key type dns_resolver registered1490lb # [ 0.835869] Freeing initrd memory: 26948K1491worker1 # [ 0.899657] registered taskstats version 11492worker2 # [ 0.841751] Freeing initrd memory: 26944K1493lb # [ 0.841854] tun: Universal TUN/TAP device driver, 1.61494worker1 # [ 0.900332] Loading compiled-in X.509 certificates1495client # [ 0.841981] Freeing initrd memory: 26972K1496worker1 # [ 0.908585] Demotion targets for Node 0: null1497worker2 # [ 0.847690] tun: Universal TUN/TAP device driver, 1.61498worker1 # [ 0.908688] Key type .fscrypt registered1499worker1 # [ 0.908695] Key type fscrypt-provisioning registered1500worker1 # [ 0.908800] ima: No TPM chip found, activating TPM-bypass!1501lb # [ 0.845553] thunder_xcv, ver 1.01502worker1 # [ 0.908819] ima: Allocated hash algorithm: sha11503lb # [ 0.845600] thunder_bgx, ver 1.01504lb # [ 0.845623] nicpf, ver 1.01505worker1 # [ 0.908841] ima: No architecture policies found1506lb # [ 0.846192] e1000: Intel(R) PRO/1000 Network Driver1507client # [ 0.847878] tun: Universal TUN/TAP device driver, 1.61508lb # [ 0.846200] e1000: Copyright (c) 1999-2006 Intel Corporation.1509worker1 # [ 0.912866] input: gpio-keys as /devices/platform/gpio-keys/input/input01510lb # [ 0.846223] e1000e: Intel(R) PRO/1000 Network Driver1511worker2 # [ 0.851365] thunder_xcv, ver 1.01512lb # [ 0.846230] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1513worker2 # [ 0.851405] thunder_bgx, ver 1.01514worker2 # [ 0.851428] nicpf, ver 1.01515lb # [ 0.846259] igb: Intel(R) Gigabit Ethernet Network Driver1516worker2 # [ 0.852006] e1000: Intel(R) PRO/1000 Network Driver1517lb # [ 0.846265] igb: Copyright (c) 2007-2014 Intel Corporation.1518worker2 # [ 0.852014] e1000: Copyright (c) 1999-2006 Intel Corporation.1519lb # [ 0.846288] igbvf: Intel(R) Gigabit Virtual Function Network Driver1520worker2 # [ 0.852037] e1000e: Intel(R) PRO/1000 Network Driver1521lb # [ 0.846294] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1522worker2 # [ 0.852044] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1523lb # [ 0.846429] sky2: driver version 1.301524worker2 # [ 0.852073] igb: Intel(R) Gigabit Ethernet Network Driver1525lb # [ 0.848036] usbcore: registered new interface driver usb-storage1526worker2 # [ 0.852079] igb: Copyright (c) 2007-2014 Intel Corporation.1527lb # [ 0.848084] usbcore: registered new interface driver usbserial_generic1528worker2 # [ 0.852101] igbvf: Intel(R) Gigabit Virtual Function Network Driver1529lb # [ 0.848103] usbserial: USB Serial support registered for generic1530lb # [ 0.849243] ehci-pci 0000:00:07.0: EHCI Host Controller1531worker2 # [ 0.852107] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1532worker2 # [ 0.852238] sky2: driver version 1.301533client # [ 0.851482] thunder_xcv, ver 1.01534lb # [ 0.849267] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11535client # [ 0.851518] thunder_bgx, ver 1.01536client # [ 0.851540] nicpf, ver 1.01537lb # [ 0.849528] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001538client # [ 0.852089] e1000: Intel(R) PRO/1000 Network Driver1539client # [ 0.852097] e1000: Copyright (c) 1999-2006 Intel Corporation.1540lb # [ 0.861059] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001541worker2 # [ 0.861077] usbcore: registered new interface driver usb-storage1542client # [ 0.852123] e1000e: Intel(R) PRO/1000 Network Driver1543lb # [ 0.861356] hub 1-0:1.0: USB hub found1544worker2 # [ 0.861773] ehci-pci 0000:00:07.0: EHCI Host Controller1545client # [ 0.852131] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1546lb # [ 0.861377] hub 1-0:1.0: 6 ports detected1547client # [ 0.852160] igb: Intel(R) Gigabit Ethernet Network Driver1548worker2 # [ 0.861799] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11549client # [ 0.852166] igb: Copyright (c) 2007-2014 Intel Corporation.1550worker2 # [ 0.862051] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001551lb # [ 0.863633] hv_vmbus: registering driver hyperv_keyboard1552client # [ 0.852188] igbvf: Intel(R) Gigabit Virtual Function Network Driver1553client # [ 0.852194] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1554client # [ 0.852350] sky2: driver version 1.301555worker2 # [ 0.865158] usbcore: registered new interface driver usbserial_generic1556lb # [ 0.865174] rtc-pl031 9010000.pl031: registered as rtc01557client # [ 0.853926] usbcore: registered new interface driver usb-storage1558worker2 # [ 0.865175] usbserial: USB Serial support registered for generic1559lb # [ 0.865199] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T10:34:14 UTC (1790159654)1560client # [ 0.854013] usbcore: registered new interface driver usbserial_generic1561worker2 # [ 0.865795] hv_vmbus: registering driver hyperv_keyboard1562lb # [ 0.865498] i2c_dev: i2c /dev entries driver1563client # [ 0.854027] usbserial: USB Serial support registered for generic1564client # [ 0.854628] hv_vmbus: registering driver hyperv_keyboard1565worker2 # [ 0.868779] rtc-pl031 9010000.pl031: registered as rtc01566client # [ 0.855936] ehci-pci 0000:00:07.0: EHCI Host Controller1567worker2 # [ 0.868806] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T10:34:14 UTC (1790159654)1568worker1 # [ 0.930154] clk: Disabling unused clocks1569client # [ 0.855960] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11570worker2 # [ 0.869110] i2c_dev: i2c /dev entries driver1571worker1 # [ 0.930173] PM: genpd: Disabling unused power domains1572client # [ 0.856248] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001573client # [ 0.867691] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001574lb # [ 0.870828] sdhci: Secure Digital Host Controller Interface driver1575client # [ 0.867970] hub 1-0:1.0: USB hub found1576worker1 # [ 0.934248] Freeing unused kernel memory: 4736K1577lb # [ 0.870838] sdhci: Copyright(c) Pierre Ossman1578client # [ 0.867993] hub 1-0:1.0: 6 ports detected1579worker1 # [ 0.934460] Run /init as init process1580lb # [ 0.871114] Synopsys Designware Multimedia Card Interface Driver1581lb # [ 0.871489] sdhci-pltfm: SDHCI platform and OF driver helper1582client # [ 0.870808] rtc-pl031 9010000.pl031: registered as rtc01583client # [ 0.870839] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T10:34:14 UTC (1790159654)1584client # [ 0.871182] i2c_dev: i2c /dev entries driver1585lb # [ 0.875702] hid: raw HID events driver (C) Jiri Kosina1586worker2 # [ 0.874471] sdhci: Secure Digital Host Controller Interface driver1587lb # [ 0.875975] usbcore: registered new interface driver usbhid1588worker2 # [ 0.874482] sdhci: Copyright(c) Pierre Ossman1589lb # [ 0.875981] usbhid: USB HID core driver1590worker2 # [ 0.874745] Synopsys Designware Multimedia Card Interface Driver1591worker2 # [ 0.875113] sdhci-pltfm: SDHCI platform and OF driver helper1592worker2 # [ 0.876604] hid: raw HID events driver (C) Jiri Kosina1593worker2 # [ 0.876840] usbcore: registered new interface driver usbhid1594worker2 # [ 0.876846] usbhid: USB HID core driver1595lb # [ 0.878920] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1596lb # [ 0.880454] drop_monitor: Initializing network drop monitor service1597worker2 # [ 0.881565] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001598worker2 # [ 0.881862] hub 1-0:1.0: USB hub found1599lb # [ 0.882445] NET: Registered PF_INET6 protocol family1600client # [ 0.876531] sdhci: Secure Digital Host Controller Interface driver1601worker2 # [ 0.881881] hub 1-0:1.0: 6 ports detected1602client # [ 0.876541] sdhci: Copyright(c) Pierre Ossman1603lb # [ 0.883606] Segment Routing with IPv61604client # [ 0.876812] Synopsys Designware Multimedia Card Interface Driver1605lb # [ 0.883628] In-situ OAM (IOAM) with IPv61606client # [ 0.877193] sdhci-pltfm: SDHCI platform and OF driver helper1607lb # [ 0.883665] NET: Registered PF_PACKET protocol family1608worker2 # [ 0.884665] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1609client # [ 0.878720] hid: raw HID events driver (C) Jiri Kosina1610client # [ 0.878963] usbcore: registered new interface driver usbhid1611client # [ 0.878970] usbhid: USB HID core driver1612worker2 # [ 0.887250] drop_monitor: Initializing network drop monitor service1613worker2 # [ 0.887376] NET: Registered PF_INET6 protocol family1614worker1 # [ 0.949815] systemd[1]: Successfully made /usr/ read-only.1615lb # [ 0.885474] 9pnet: Installing 9P2000 support1616worker2 # [ 0.889237] Segment Routing with IPv61617lb # [ 0.885516] Key type dns_resolver registered1618worker2 # [ 0.889253] In-situ OAM (IOAM) with IPv61619worker2 # [ 0.889281] NET: Registered PF_PACKET protocol family1620client # [ 0.884566] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1621client # [ 0.886112] drop_monitor: Initializing network drop monitor service1622client # [ 0.886263] NET: Registered PF_INET6 protocol family1623lb # [ 0.892302] registered taskstats version 11624client # [ 0.889239] Segment Routing with IPv61625worker2 # [ 0.890914] 9pnet: Installing 9P2000 support1626lb # [ 0.892460] Loading compiled-in X.509 certificates1627client # [ 0.889258] In-situ OAM (IOAM) with IPv61628worker2 # [ 0.890958] Key type dns_resolver registered1629client # [ 0.889289] NET: Registered PF_PACKET protocol family1630client # [ 0.890899] 9pnet: Installing 9P2000 support1631client # [ 0.890940] Key type dns_resolver registered1632worker2 # [ 0.897550] registered taskstats version 11633lb # [ 0.900829] Demotion targets for Node 0: null1634worker2 # [ 0.897695] Loading compiled-in X.509 certificates1635lb # [ 0.900925] Key type .fscrypt registered1636lb # [ 0.900931] Key type fscrypt-provisioning registered1637client # [ 0.897822] registered taskstats version 11638lb # [ 0.901017] ima: No TPM chip found, activating TPM-bypass!1639client # [ 0.897991] Loading compiled-in X.509 certificates1640lb # [ 0.901036] ima: Allocated hash algorithm: sha11641lb # [ 0.901056] ima: No architecture policies found1642lb # [ 0.905069] input: gpio-keys as /devices/platform/gpio-keys/input/input01643worker2 # [ 0.906281] Demotion targets for Node 0: null1644worker2 # [ 0.906373] Key type .fscrypt registered1645worker2 # [ 0.906379] Key type fscrypt-provisioning registered1646worker2 # [ 0.906473] ima: No TPM chip found, activating TPM-bypass!1647worker2 # [ 0.906492] ima: Allocated hash algorithm: sha11648worker2 # [ 0.906512] ima: No architecture policies found1649worker2 # [ 0.910454] input: gpio-keys as /devices/platform/gpio-keys/input/input01650client # [ 0.906695] Demotion targets for Node 0: null1651client # [ 0.906797] Key type .fscrypt registered1652client # [ 0.906803] Key type fscrypt-provisioning registered1653client # [ 0.906889] ima: No TPM chip found, activating TPM-bypass!1654client # [ 0.906908] ima: Allocated hash algorithm: sha11655client # [ 0.906935] ima: No architecture policies found1656client # [ 0.910880] input: gpio-keys as /devices/platform/gpio-keys/input/input01657lb # [ 0.922375] clk: Disabling unused clocks1658lb # [ 0.922396] PM: genpd: Disabling unused power domains1659lb # [ 0.926482] Freeing unused kernel memory: 4736K1660lb # [ 0.926685] Run /init as init process1661worker2 # [ 0.928050] clk: Disabling unused clocks1662worker2 # [ 0.928069] PM: genpd: Disabling unused power domains1663worker2 # [ 0.932133] Freeing unused kernel memory: 4736K1664worker2 # [ 0.932330] Run /init as init process1665client # [ 0.928651] clk: Disabling unused clocks1666client # [ 0.928675] PM: genpd: Disabling unused power domains1667client # [ 0.932802] Freeing unused kernel memory: 4736K1668client # [ 0.933001] Run /init as init process1669lb # [ 0.942588] systemd[1]: Successfully made /usr/ read-only.1670worker2 # [ 0.947232] systemd[1]: Successfully made /usr/ read-only.1671client # [ 0.949381] systemd[1]: Successfully made /usr/ read-only.1672worker1 # [ 1.131782] usb 1-1: new high-speed USB device number 2 using ehci-pci1673lb # [ 1.108568] usb 1-1: new high-speed USB device number 2 using ehci-pci1674client # [ 1.115363] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker2 # [ 1.129191] usb 1-1: new high-speed USB device number 2 using ehci-pci1676worker1 # [ 1.284217] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677worker1 # [ 1.290309] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678worker1 # [ 1.302698] systemd[1]: Detected virtualization qemu.1679worker1 # [ 1.304838] systemd[1]: Detected architecture arm64.1680worker1 # [ 1.306747] systemd[1]: Running in initrd.1681worker1 # [ 1.309389] systemd[1]: Initializing machine ID from random generator.1682worker1 # [ 1.312252] systemd[1]: Hostname set to <worker1>.1683lb # [ 1.260946] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684client # [ 1.267824] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685worker2 # [ 1.281670] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11686lb # [ 1.277371] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1687lb # [ 1.289420] systemd[1]: Detected virtualization qemu.1688lb # [ 1.291402] systemd[1]: Detected architecture arm64.1689lb # [ 1.293379] systemd[1]: Running in initrd.1690lb # [ 1.295912] systemd[1]: Initializing machine ID from random generator.1691client # [ 1.284059] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692worker2 # [ 1.287776] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1693lb # [ 1.298802] systemd[1]: Hostname set to <lb>.1694worker2 # [ 1.300259] systemd[1]: Detected virtualization qemu.1695client # [ 1.296100] systemd[1]: Detected virtualization qemu.1696worker2 # [ 1.302403] systemd[1]: Detected architecture arm64.1697client # [ 1.296181] systemd[1]: Detected architecture arm64.1698worker2 # [ 1.304307] systemd[1]: Running in initrd.1699client # [ 1.296207] systemd[1]: Running in initrd.1700client # [ 1.297081] systemd[1]: Initializing machine ID from random generator.1701worker2 # [ 1.306899] systemd[1]: Initializing machine ID from random generator.1702worker1 # [ 1.367934] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01703client # [ 1.297344] systemd[1]: Hostname set to <client>.1704worker2 # [ 1.309781] systemd[1]: Hostname set to <worker2>.1705lb # [ 1.344787] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706client # [ 1.351538] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker2 # [ 1.365345] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708worker1 # [ 1.487789] usb 1-2: new high-speed USB device number 3 using ehci-pci1709lb # [ 1.468542] usb 1-2: new high-speed USB device number 3 using ehci-pci1710client # [ 1.471356] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker2 # [ 1.485160] usb 1-2: new high-speed USB device number 3 using ehci-pci1712worker1 # [ 1.615890] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713lb # [ 1.592044] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714worker1 # [ 1.651576] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21715worker1 # [ 1.657229] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01716client # [ 1.598664] systemd[1]: bpf-restrict-fs: LSM BPF program attached1717worker2 # [ 1.608291] systemd[1]: bpf-restrict-fs: LSM BPF program attached1718lb # [ 1.623043] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719lb # [ 1.629107] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720client # [ 1.626035] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721client # [ 1.631721] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722worker2 # [ 1.648839] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21723worker2 # [ 1.649232] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01724worker1 # [ 1.734945] systemd[1]: Queued start job for default target Initrd Default Target.1725worker1 # [ 1.746149] systemd[1]: Created slice Slice /system/modprobe.1726worker1 # [ 1.747379] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1727worker1 # [ 1.748845] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728worker1 # [ 1.749964] systemd[1]: Reached target Path Units.1729worker1 # [ 1.750815] systemd[1]: Reached target Slice Units.1730worker1 # [ 1.751681] systemd[1]: Reached target Swaps.1731worker1 # [ 1.752540] systemd[1]: Reached target Timer Units.1732worker1 # [ 1.753593] systemd[1]: Listening on D-Bus System Message Bus Socket.1733worker1 # [ 1.754864] systemd[1]: Listening on Journal Socket (/dev/log).1734worker1 # [ 1.756073] systemd[1]: Listening on Journal Sockets.1735worker1 # [ 1.756219] systemd[1]: Listening on udev Control Socket.1736worker1 # [ 1.756338] systemd[1]: Listening on udev Kernel Socket.1737worker1 # [ 1.756360] systemd[1]: Reached target Socket Units.1738lb # [ 1.698649] systemd[1]: Queued start job for default target Initrd Default Target.1739worker1 # [ 1.761508] systemd[1]: Starting Create List of Static Device Nodes...1740worker1 # [ 1.762705] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1741lb # [ 1.706908] systemd[1]: Created slice Slice /system/modprobe.1742lb # [ 1.708088] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1743lb # [ 1.709492] systemd[1]: Expecting device /dev/disk/by-label/nixos...1744worker1 # [ 1.771881] systemd[1]: Mounting Kernel Configuration File System...1745lb # [ 1.710548] systemd[1]: Reached target Path Units.1746lb # [ 1.711377] systemd[1]: Reached target Slice Units.1747lb # [ 1.712206] systemd[1]: Reached target Swaps.1748lb # [ 1.712997] systemd[1]: Reached target Timer Units.1749client # [ 1.709766] systemd[1]: Queued start job for default target Initrd Default Target.1750lb # [ 1.713994] systemd[1]: Listening on D-Bus System Message Bus Socket.1751lb # [ 1.715202] systemd[1]: Listening on Journal Socket (/dev/log).1752lb # [ 1.716329] systemd[1]: Listening on Journal Sockets.1753worker1 # [ 1.779987] systemd[1]: Starting Journal Service...1754lb # [ 1.717373] systemd[1]: Listening on udev Control Socket.1755lb # [ 1.717489] systemd[1]: Listening on udev Kernel Socket.1756lb # [ 1.717514] systemd[1]: Reached target Socket Units.1757client # [ 1.717398] systemd[1]: Created slice Slice /system/modprobe.1758lb # [ 1.721575] systemd[1]: Starting Create List of Static Device Nodes...1759client # [ 1.718457] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1760client # [ 1.719719] systemd[1]: Expecting device /dev/disk/by-label/nixos...1761lb # [ 1.722721] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1762client # [ 1.720776] systemd[1]: Reached target Path Units.1763worker2 # [ 1.724235] systemd[1]: Queued start job for default target Initrd Default Target.1764client # [ 1.721497] systemd[1]: Reached target Slice Units.1765client # [ 1.722226] systemd[1]: Reached target Swaps.1766client # [ 1.722876] systemd[1]: Reached target Timer Units.1767client # [ 1.723812] systemd[1]: Listening on D-Bus System Message Bus Socket.1768client # [ 1.724906] systemd[1]: Listening on Journal Socket (/dev/log).1769client # [ 1.725888] systemd[1]: Listening on Journal Sockets.1770client # [ 1.726751] systemd[1]: Listening on udev Control Socket.1771lb # [ 1.730117] systemd[1]: Mounting Kernel Configuration File System...1772client # [ 1.727722] systemd[1]: Listening on udev Kernel Socket.1773client # [ 1.727748] systemd[1]: Reached target Socket Units.1774client # [ 1.730606] systemd[1]: Starting Create List of Static Device Nodes...1775worker2 # [ 1.735547] systemd[1]: Created slice Slice /system/modprobe.1776client # [ 1.731645] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1777worker2 # [ 1.736798] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1778worker2 # [ 1.738232] systemd[1]: Expecting device /dev/disk/by-label/nixos...1779worker2 # [ 1.739313] systemd[1]: Reached target Path Units.1780worker2 # [ 1.740148] systemd[1]: Reached target Slice Units.1781worker2 # [ 1.740995] systemd[1]: Reached target Swaps.1782worker2 # [ 1.741854] systemd[1]: Reached target Timer Units.1783worker1 # [ 1.803545] systemd[1]: Starting Load Kernel Modules...1784worker2 # [ 1.742893] systemd[1]: Listening on D-Bus System Message Bus Socket.1785worker2 # [ 1.744165] systemd[1]: Listening on Journal Socket (/dev/log).1786worker1 # [ 1.804619] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1787client # [ 1.741948] systemd[1]: Mounting Kernel Configuration File System...1788lb # [ 1.744935] systemd[1]: Starting Journal Service...1789worker2 # [ 1.745362] systemd[1]: Listening on Journal Sockets.1790worker2 # [ 1.745513] systemd[1]: Listening on udev Control Socket.1791worker2 # [ 1.745631] systemd[1]: Listening on udev Kernel Socket.1792worker2 # [ 1.745655] systemd[1]: Reached target Socket Units.1793worker2 # [ 1.750753] systemd[1]: Starting Create List of Static Device Nodes...1794worker2 # [ 1.751950] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1795lb # [ 1.752666] systemd[1]: Starting Load Kernel Modules...1796lb # [ 1.752780] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1797client # [ 1.751444] systemd[1]: Starting Journal Service...1798worker1 # [ 1.819874] systemd[1]: Starting Coldplug All udev Devices...1799worker2 # [ 1.761273] systemd[1]: Mounting Kernel Configuration File System...1800worker2 # [ 1.769399] systemd[1]: Starting Journal Service...1801worker1 # [ 1.833353] systemd-journald[72]: Collecting audit messages is disabled.1802client # [ 1.767795] systemd[1]: Starting Load Kernel Modules...1803client # [ 1.767904] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1804worker1 # [ 1.839885] systemd[1]: Finished Create List of Static Device Nodes.1805worker1 # [ 1.840524] systemd[1]: Mounted Kernel Configuration File System.1806lb # [ 1.784105] systemd[1]: Starting Coldplug All udev Devices...1807worker1 # [ 1.847932] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1808client # [ 1.787509] systemd[1]: Starting Coldplug All udev Devices...1809worker2 # [ 1.793083] systemd[1]: Starting Load Kernel Modules...1810worker2 # [ 1.794177] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1811client # [ 1.793525] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1812lb # [ 1.796651] systemd[1]: Finished Create List of Static Device Nodes.1813lb # [ 1.804952] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1814client # [ 1.804701] systemd[1]: Finished Create List of Static Device Nodes.1815client # [ 1.805345] systemd[1]: Mounted Kernel Configuration File System.1816client # [ 1.811700] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1817worker2 # [ 1.815290] systemd[1]: Starting Coldplug All udev Devices...1818worker2 # [ 1.818019] systemd-journald[72]: Collecting audit messages is disabled.1819lb # [ 1.818205] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1820worker1 # [ 1.882508] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1821client # [ 1.819352] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1822worker1 # [ 1.886841] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1823lb # [ 1.820613] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1824lb # [ 1.826651] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01825lb # [ 1.826874] [drm] features: -virgl +edid -resource_blob -host_visible1826lb # [ 1.826882] [drm] features: -context_init1827worker2 # [ 1.829255] systemd[1]: Finished Create List of Static Device Nodes.1828lb # [ 1.827559] [drm] number of scanouts: 11829lb # [ 1.827576] [drm] number of cap sets: 01830worker2 # [ 1.829917] systemd[1]: Mounted Kernel Configuration File System.1831worker1 # [ 1.892335] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1832client # [ 1.828461] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01833worker1 # [ 1.894765] systemd[1]: Starting Create Static Device Nodes in /dev...1834client # [ 1.828696] [drm] features: -virgl +edid -resource_blob -host_visible1835client # [ 1.828705] [drm] features: -context_init1836client # [ 1.829398] [drm] number of scanouts: 11837client # [ 1.829415] [drm] number of cap sets: 01838lb # [ 1.844093] systemd[1]: Mounted Kernel Configuration File System.1839worker2 # [ 1.841503] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1840client # [ 1.842467] systemd-journald[72]: Collecting audit messages is disabled.1841worker1 # [ 1.912124] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01842worker1 # [ 1.912365] [drm] features: -virgl +edid -resource_blob -host_visible1843lb # [ 1.850930] systemd-journald[72]: Collecting audit messages is disabled.1844worker1 # [ 1.912375] [drm] features: -context_init1845worker1 # [ 1.913068] [drm] number of scanouts: 11846lb # [ 1.851490] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1847lb # [ 1.854664] systemd[1]: Starting Create Static Device Nodes in /dev...1848worker1 # [ 1.913086] [drm] number of cap sets: 01849lb # [ 1.856868] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1850client # [ 1.851611] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1851lb # [ 1.856883] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01852client # [ 1.851627] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01853worker1 # [ 1.924069] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1854worker1 # [ 1.924085] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01855client # [ 1.871834] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1856client # [ 1.874348] systemd[1]: Starting Create Static Device Nodes in /dev...1857worker2 # [ 1.881757] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1858worker1 # [ 1.944320] systemd[1]: Finished Create Static Device Nodes in /dev.1859worker2 # [ 1.884214] systemd[1]: Starting Create Static Device Nodes in /dev...1860worker1 # [ 1.944646] systemd[1]: Reached target Preparation for Local File Systems.1861worker1 # [ 1.944673] systemd[1]: Reached target Local File Systems.1862lb # [ 1.880786] Console: switching to colour frame buffer device 160x501863worker1 # [ 1.950778] systemd[1]: Starting Rule-based Manager for Device Events and Files...1864client # [ 1.881981] Console: switching to colour frame buffer device 160x501865worker2 # [ 1.892587] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1866worker1 # [ 1.956105] Console: switching to colour frame buffer device 160x501867lb # [ 1.909137] systemd[1]: Finished Create Static Device Nodes in /dev.1868lb # [ 1.909294] systemd[1]: Reached target Preparation for Local File Systems.1869client # [ 1.907909] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1870lb # [ 1.909318] systemd[1]: Reached target Local File Systems.1871lb # [ 1.913412] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1872lb # [ 1.916238] systemd[1]: Starting Rule-based Manager for Device Events and Files...1873worker1 # [ 1.980389] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1874worker2 # [ 1.909733] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1875worker2 # [ 1.915766] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01876client # [ 1.916560] systemd[1]: Finished Create Static Device Nodes in /dev.1877worker2 # [ 1.915999] [drm] features: -virgl +edid -resource_blob -host_visible1878client # [ 1.916899] systemd[1]: Reached target Preparation for Local File Systems.1879worker2 # [ 1.916009] [drm] features: -context_init1880client # [ 1.916937] systemd[1]: Reached target Local File Systems.1881worker2 # [ 1.916711] [drm] number of scanouts: 11882worker2 # [ 1.916729] [drm] number of cap sets: 01883client # [ 1.921689] systemd[1]: Starting Rule-based Manager for Device Events and Files...1884lb # [ 1.929012] systemd[1]: Finished Load Kernel Modules.1885worker1 # [ 1.996944] systemd[1]: Finished Load Kernel Modules.1886worker2 # [ 1.937752] systemd[1]: Finished Create Static Device Nodes in /dev.1887worker2 # [ 1.938075] systemd[1]: Reached target Preparation for Local File Systems.1888worker2 # [ 1.938104] systemd[1]: Reached target Local File Systems.1889worker1 # [ 2.004154] systemd[1]: Starting Apply Kernel Variables...1890worker2 # [ 1.941896] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1891lb # [ 1.940911] systemd[1]: Starting Apply Kernel Variables...1892worker2 # [ 1.941910] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01893client # [ 1.943377] systemd[1]: Finished Load Kernel Modules.1894client # [ 1.946263] systemd[1]: Starting Apply Kernel Variables...1895worker1 # [ 1.997213] systemd-modules-load[74]: Inserted module 'dm_mod'1896worker1 # [ 1.998272] systemd-modules-load[74]: Module 'virtio_balloon' is built in1897worker2 # [ 1.949626] systemd[1]: Starting Rule-based Manager for Device Events and Files...1898worker1 # [ 2.014982] systemd[1]: Started Journal Service.1899worker1 # [ 2.003541] systemd-modules-load[74]: Module 'virtio_console' is built in1900worker1 # [ 2.012338] systemd-modules-load[74]: Inserted module 'virtio_gpu'1901worker1 # [ 2.013359] systemd-modules-load[74]: Module 'virtio_rng' is built in1902worker2 # [ 1.969580] Console: switching to colour frame buffer device 160x501903lb # [ 1.977098] systemd[1]: Finished Apply Kernel Variables.1904worker1 # [ 2.024165] systemd[1]: Starting Create System Files and Directories...1905worker1 # [ 2.038152] systemd[1]: Finished Apply Kernel Variables.1906client # [ 1.987949] systemd[1]: Finished Apply Kernel Variables.1907worker2 # [ 1.996668] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1908worker1 # [ 2.047538] systemd-udevd[79]: Using default interface naming scheme 'v261'.1909worker2 # [ 2.002002] systemd[1]: Started Journal Service.1910worker2 # [ 1.987356] systemd-modules-load[74]: Inserted module 'dm_mod'1911worker2 # [ 1.992762] systemd-modules-load[74]: Module 'virtio_balloon' is built in1912client # [ 2.005596] systemd[1]: Started Journal Service.1913lb # [ 2.008096] systemd[1]: Started Journal Service.1914worker2 # [ 1.993869] systemd-modules-load[74]: Module 'virtio_console' is built in1915worker2 # [ 1.994922] systemd-modules-load[74]: Inserted module 'virtio_gpu'1916worker2 # [ 1.995907] systemd-modules-load[74]: Module 'virtio_rng' is built in1917lb # [ 2.004355] systemd-modules-load[73]: Inserted module 'dm_mod'1918lb # [ 2.005436] systemd-modules-load[73]: Module 'virtio_balloon' is built in1919client # [ 2.004280] systemd-modules-load[73]: Inserted module 'dm_mod'1920lb # [ 2.007311] systemd-modules-load[73]: Module 'virtio_console' is built in1921client # [ 2.005371] systemd-modules-load[73]: Module 'virtio_balloon' is built in1922client # [ 2.006444] systemd-modules-load[73]: Module 'virtio_console' is built in1923client # [ 2.007478] systemd-modules-load[73]: Inserted module 'virtio_gpu'1924worker1 # [ 2.073504] systemd[1]: Finished Create System Files and Directories.1925worker2 # [ 2.013491] systemd[1]: Finished Load Kernel Modules.1926client # [ 2.013419] systemd-modules-load[73]: Module 'virtio_rng' is built in1927worker2 # [ 2.015508] systemd[1]: Starting Apply Kernel Variables...1928lb # [ 2.017358] systemd-modules-load[73]: Inserted module 'virtio_gpu'1929lb # [ 2.021175] systemd-modules-load[73]: Module 'virtio_rng' is built in1930worker1 # [ 2.086341] systemd[1]: Started Rule-based Manager for Device Events and Files.1931worker2 # [ 2.023658] systemd[1]: Starting Create System Files and Directories...1932lb # [ 2.025644] systemd[1]: Starting Create System Files and Directories...1933worker2 # [ 2.028327] systemd-udevd[79]: Using default interface naming scheme 'v261'.1934lb # [ 2.029705] systemd-udevd[79]: Using default interface naming scheme 'v261'.1935client # [ 2.028340] systemd[1]: Starting Create System Files and Directories...1936client # [ 2.029686] systemd-udevd[79]: Using default interface naming scheme 'v261'.1937client # [ 2.037189] systemd[1]: Finished Create System Files and Directories.1938lb # [ 2.043859] systemd[1]: Finished Create System Files and Directories.1939lb # [ 2.061205] systemd[1]: Started Rule-based Manager for Device Events and Files.1940client # [ 2.061455] systemd[1]: Started Rule-based Manager for Device Events and Files.1941worker2 # [ 2.064840] systemd[1]: Finished Apply Kernel Variables.1942worker2 # [ 2.075325] systemd[1]: Started Rule-based Manager for Device Events and Files.1943worker2 # [ 2.076988] systemd[1]: Finished Create System Files and Directories.1944worker1 # [ 2.145506] systemd[1]: Starting Virtual Console Setup...1945client # [ 2.116227] systemd[1]: Starting Virtual Console Setup...1946lb # [ 2.120056] systemd[1]: Starting Virtual Console Setup...1947worker2 # [ 2.128093] systemd[1]: Starting Virtual Console Setup...1948worker1 # [ 2.196441] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1949worker1 # [ 2.199653] systemd[1]: Finished Virtual Console Setup.1950client # [ 2.160446] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1951lb # [ 2.164454] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1952client # [ 2.163700] systemd[1]: Finished Virtual Console Setup.1953lb # [ 2.167688] systemd[1]: Finished Virtual Console Setup.1954worker2 # [ 2.176442] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1955worker2 # [ 2.179617] systemd[1]: Finished Virtual Console Setup.1956worker1 # [ 2.813079] systemd[1]: Finished Coldplug All udev Devices.1957worker1 # [ 2.814107] systemd[1]: Reached target System Initialization.1958worker1 # [ 2.814932] systemd[1]: Reached target Basic System.1959lb # [ 2.761274] systemd[1]: Finished Coldplug All udev Devices.1960lb # [ 2.762238] systemd[1]: Reached target System Initialization.1961lb # [ 2.763024] systemd[1]: Reached target Basic System.1962client # [ 2.761124] systemd[1]: Finished Coldplug All udev Devices.1963client # [ 2.762440] systemd[1]: Reached target System Initialization.1964client # [ 2.764490] systemd[1]: Reached target Basic System.1965worker2 # [ 2.776400] systemd[1]: Finished Coldplug All udev Devices.1966worker2 # [ 2.777583] systemd[1]: Reached target System Initialization.1967worker2 # [ 2.778499] systemd[1]: Reached target Basic System.1968worker1 # [ 2.948206] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969lb # [ 2.907713] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1970worker2 # [ 2.920345] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971worker1 # [ 2.983338] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1972worker1 # [ 2.993876] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1973client # [ 2.930720] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1974client # [ 2.950413] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975client # [ 2.954537] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1976lb # [ 2.957999] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1977worker2 # [ 2.962381] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1978lb # [ 2.964251] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1979worker2 # [ 2.964851] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1980client # [ 3.008087] systemd[1]: Found device /dev/disk/by-label/nixos.1981client # [ 3.012610] systemd[1]: Reached target Initrd Root Device.1982worker1 # [ 3.080396] systemd[1]: Found device /dev/disk/by-label/nixos.1983client # [ 3.016113] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1984worker1 # [ 3.084193] systemd[1]: Reached target Initrd Root Device.1985worker1 # [ 3.089206] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986lb # [ 3.035490] systemd[1]: Found device /dev/disk/by-label/nixos.1987lb # [ 3.038427] systemd[1]: Reached target Initrd Root Device.1988lb # [ 3.044148] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989worker2 # [ 3.044994] systemd[1]: Found device /dev/disk/by-label/nixos.1990worker2 # [ 3.049875] systemd[1]: Reached target Initrd Root Device.1991worker2 # [ 3.056108] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992worker1 # [ 3.132449] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1993client # [ 3.069643] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1994client # [ 3.075040] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995worker1 # [ 3.139593] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1996worker1 # [ 3.141125] systemd[1]: Mounting /sysroot...1997lb # [ 3.083408] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1998client # [ 3.087499] systemd[1]: Mounting /sysroot...1999lb # [ 3.091242] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000worker2 # [ 3.095509] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 2,100 hidden lines…
4101worker1 # [ 57.636335] nix-grpc-daemon[486]: ts=2026-09-23T10:35:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_in=5124102lb # [ 57.577120] niks3-server[977]: 2026/09/23 10:35:11 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4103worker1 # [ 57.651845] nix-grpc-daemon[486]: ts=2026-09-23T10:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344104worker1 # [ 57.655235] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4105worker1 # [ 57.661517] nix-daemon[851]: reaped child process 900, status = succeeded4106client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4107worker1 # [ 57.672923] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4108client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4109client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4110client # farm-a> LOG-farm-a-sig4111lb # [ 58.356202] envoy[895]: conn peer=[2001:db8:1::1]:50178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9562 tx=4392 ms=60164112lb # [ 58.356522] envoy[895]: conn peer=[2001:db8:1::1]:40028 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=734113lb # [ 58.356657] envoy[895]: conn peer=[2001:db8:1::1]:40044 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1898 ms=18514114worker1 # [ 58.548309] nix-daemon[851]: reaped child process 921, status = succeeded4115worker1 # [ 58.646281] nix-daemon[851]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4116worker1 # [ 58.666409] nix-daemon[851]: reaped child process 940, status = succeeded4117lb # [ 58.603426] niks3-server[977]: 2026/09/23 10:35:12 INFO Received uploads request method=POST path=/api/pending_closures4118worker1 # [ 58.676488] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.319Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4119worker1 # [ 58.679748] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.319Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4120worker1 # [ 58.705589] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.350Z level=INFO msg="Uploading 1 narinfos"4121lb # [ 58.642469] niks3-server[977]: 2026/09/23 10:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4122lb # [ 58.660185] niks3-server[977]: 2026/09/23 10:35:12 INFO Signed narinfos id=10 count=14123lb # [ 58.668425] niks3-server[977]: 2026/09/23 10:35:12 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4124lb # [ 58.676746] niks3-server[977]: 2026/09/23 10:35:12 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4125worker1 # [ 58.748536] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.393Z level=INFO msg="Upload complete. (266ms)"4126lb # [ 58.688845] niks3-server[977]: 2026/09/23 10:35:12 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4127lb # [ 58.697068] niks3-server[977]: 2026/09/23 10:35:12 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4128lb # [ 58.701526] niks3-server[977]: 2026/09/23 10:35:12 INFO Completed upload id=104129worker1 # [ 58.762374] nix-grpc-daemon[486]: ts=2026-09-23T10:35:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14130lb # [ 58.703790] niks3-server[977]: 2026/09/23 10:35:12 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4131worker1 # [ 58.783639] nix-grpc-daemon[486]: ts=2026-09-23T10:35:12Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04132worker1 # [ 58.789249] nix-grpc-daemon[486]: ts=2026-09-23T10:35:12Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14133worker1 # [ 58.795152] nix-grpc-daemon[486]: ts=2026-09-23T10:35:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344134worker1 # [ 58.800571] nix-grpc-daemon[486]: ts=2026-09-23T10:35:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344135worker1 # [ 58.805421] nix-daemon[851]: reaped child process 919, status = succeeded4136worker1 # [ 58.808078] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.453Z level=INFO msg="All 1 paths already cached"4137client # copying 1 paths to 'lb:50051'...4138client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4139worker1 # [ 58.816759] nix-grpc-daemon[486]: ts=2026-09-23T10:35:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:469344140worker1 # [ 58.826772] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4141worker1 # [ 58.958696] nix-daemon[851]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4142worker1 # [ 58.978016] nix-daemon[851]: reaped child process 949, status = succeeded4143lb # [ 58.919940] niks3-server[977]: 2026/09/23 10:35:12 INFO Received uploads request method=POST path=/api/pending_closures4144worker1 # [ 58.991208] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.636Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4145worker1 # [ 58.992825] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.636Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4146lb # [ 58.948174] niks3-server[977]: 2026/09/23 10:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4147worker1 # [ 59.018049] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.662Z level=INFO msg="Uploading 1 narinfos"4148lb # [ 58.964046] niks3-server[977]: 2026/09/23 10:35:12 INFO Signed narinfos id=11 count=14149lb # [ 58.967171] niks3-server[977]: 2026/09/23 10:35:12 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4150worker1 # [ 59.044592] nix-grpc-daemon[902]: time=2026-09-23T10:35:12.689Z level=INFO msg="Upload complete. (206ms)"4151lb # [ 58.982877] niks3-server[977]: 2026/09/23 10:35:12 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4152lb # [ 58.987748] niks3-server[977]: 2026/09/23 10:35:12 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4153lb # [ 58.994716] niks3-server[977]: 2026/09/23 10:35:12 INFO Completed upload id=114154worker1 # [ 59.052135] nix-grpc-daemon[486]: ts=2026-09-23T10:35:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_in=6244155lb # [ 58.997575] niks3-server[977]: 2026/09/23 10:35:12 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4156worker1 # [ 59.060878] nix-grpc-daemon[486]: ts=2026-09-23T10:35:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344157worker1 # [ 59.072300] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4158worker1 # [ 59.075007] nix-daemon[851]: reaped child process 944, status = succeeded4159worker1 # [ 59.084880] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4160client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4161client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4162client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4163client # farm-b> LOG-farm-b-sig4164worker1 # [ 60.156881] nix-daemon[851]: reaped child process 953, status = succeeded4165worker1 # [ 60.266882] nix-daemon[851]: accepted connection from pid 969, user nix-grpc-daemon (trusted)4166worker1 # [ 60.285067] nix-daemon[851]: reaped child process 972, status = succeeded4167lb # [ 60.229256] niks3-server[977]: 2026/09/23 10:35:13 INFO Received uploads request method=POST path=/api/pending_closures4168worker1 # [ 60.302101] nix-grpc-daemon[902]: time=2026-09-23T10:35:13.946Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4169worker1 # [ 60.303826] nix-grpc-daemon[902]: time=2026-09-23T10:35:13.946Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4170lb # [ 60.268780] niks3-server[977]: 2026/09/23 10:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4171worker1 # [ 60.338851] nix-grpc-daemon[902]: time=2026-09-23T10:35:13.983Z level=INFO msg="Uploading 1 narinfos"4172lb # [ 60.282857] niks3-server[977]: 2026/09/23 10:35:13 INFO Signed narinfos id=12 count=14173lb # [ 60.290578] niks3-server[977]: 2026/09/23 10:35:13 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4174lb # [ 60.301193] niks3-server[977]: 2026/09/23 10:35:13 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4175worker1 # [ 60.363890] nix-grpc-daemon[902]: time=2026-09-23T10:35:14.009Z level=INFO msg="Upload complete. (267ms)"4176lb # [ 60.308367] niks3-server[977]: 2026/09/23 10:35:13 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4177lb # [ 60.311731] niks3-server[977]: 2026/09/23 10:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4178lb # [ 60.316845] niks3-server[977]: 2026/09/23 10:35:13 INFO Completed upload id=124179lb # [ 60.318467] niks3-server[977]: 2026/09/23 10:35:13 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4180worker1 # [ 60.378649] nix-grpc-daemon[486]: ts=2026-09-23T10:35:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14181worker1 # [ 60.397764] nix-grpc-daemon[486]: ts=2026-09-23T10:35:14Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04182worker1 # [ 60.403317] nix-grpc-daemon[486]: ts=2026-09-23T10:35:14Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14183worker1 # [ 60.409273] nix-grpc-daemon[486]: ts=2026-09-23T10:35:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344184worker1 # [ 60.414508] nix-grpc-daemon[486]: ts=2026-09-23T10:35:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344185worker1 # [ 60.419439] nix-daemon[851]: reaped child process 951, status = succeeded4186worker1 # [ 60.421914] nix-grpc-daemon[902]: time=2026-09-23T10:35:14.067Z level=INFO msg="All 2 paths already cached"4187client # copying 1 paths to 'lb:50051'...4188client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4189worker1 # [ 60.430639] nix-grpc-daemon[486]: ts=2026-09-23T10:35:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:469344190worker1 # [ 60.440900] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4191worker1 # [ 60.576128] nix-daemon[851]: accepted connection from pid 978, user nix-grpc-daemon (trusted)4192worker1 # [ 60.593261] nix-daemon[851]: reaped child process 981, status = succeeded4193lb # [ 60.535963] niks3-server[977]: 2026/09/23 10:35:14 INFO Received uploads request method=POST path=/api/pending_closures4194worker1 # [ 60.607564] nix-grpc-daemon[902]: time=2026-09-23T10:35:14.252Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4195worker1 # [ 60.609735] nix-grpc-daemon[902]: time=2026-09-23T10:35:14.252Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4196worker1 # [ 60.627946] nix-grpc-daemon[902]: time=2026-09-23T10:35:14.272Z level=INFO msg="Uploading 1 narinfos"4197lb # [ 60.565082] niks3-server[977]: 2026/09/23 10:35:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4198lb # [ 60.576349] niks3-server[977]: 2026/09/23 10:35:14 INFO Signed narinfos id=13 count=14199lb # [ 60.586220] niks3-server[977]: 2026/09/23 10:35:14 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4200worker1 # [ 60.669339] nix-grpc-daemon[902]: time=2026-09-23T10:35:14.314Z level=INFO msg="Upload complete. (216ms)"4201lb # [ 60.593412] niks3-server[977]: 2026/09/23 10:35:14 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4202lb # [ 60.611030] niks3-server[977]: 2026/09/23 10:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4203lb # [ 60.616827] niks3-server[977]: 2026/09/23 10:35:14 INFO Completed upload id=134204worker1 # [ 60.680253] nix-grpc-daemon[486]: ts=2026-09-23T10:35:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_in=7524205lb # [ 60.620509] niks3-server[977]: 2026/09/23 10:35:14 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4206worker1 # [ 60.691202] nix-grpc-daemon[486]: ts=2026-09-23T10:35:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344207worker1 # [ 60.701252] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4208worker1 # [ 60.705868] nix-daemon[851]: reaped child process 976, status = succeeded4209client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4210client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4211worker1 # [ 60.719787] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4212client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4213client # farm-top> LOG-farm-top-sig4214worker1 # [ 61.978777] nix-daemon[851]: reaped child process 985, status = succeeded4215worker1 # [ 62.101658] nix-daemon[851]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4216worker1 # [ 62.120679] nix-daemon[851]: reaped child process 1004, status = succeeded4217lb # [ 62.057595] niks3-server[977]: 2026/09/23 10:35:15 INFO Received uploads request method=POST path=/api/pending_closures4218worker1 # [ 62.131455] nix-grpc-daemon[902]: time=2026-09-23T10:35:15.774Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4219worker1 # [ 62.133256] nix-grpc-daemon[902]: time=2026-09-23T10:35:15.774Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4220lb # [ 62.097313] niks3-server[977]: 2026/09/23 10:35:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4221worker1 # [ 62.168545] nix-grpc-daemon[902]: time=2026-09-23T10:35:15.813Z level=INFO msg="Uploading 1 narinfos"4222lb # [ 62.111764] niks3-server[977]: 2026/09/23 10:35:15 INFO Signed narinfos id=14 count=14223lb # [ 62.119554] niks3-server[977]: 2026/09/23 10:35:15 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4224lb # [ 62.131261] niks3-server[977]: 2026/09/23 10:35:15 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4225worker1 # [ 62.204833] nix-grpc-daemon[902]: time=2026-09-23T10:35:15.849Z level=INFO msg="Upload complete. (274ms)"4226lb # [ 62.142747] niks3-server[977]: 2026/09/23 10:35:15 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4227lb # [ 62.148158] niks3-server[977]: 2026/09/23 10:35:15 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4228lb # [ 62.153607] niks3-server[977]: 2026/09/23 10:35:15 INFO Completed upload id=144229lb # [ 62.156625] niks3-server[977]: 2026/09/23 10:35:15 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4230worker1 # [ 62.217506] nix-grpc-daemon[486]: ts=2026-09-23T10:35:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14231worker1 # [ 62.238166] nix-daemon[851]: reaped child process 983, status = succeeded4232client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.38 seconds)4233client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4234client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4235client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4236worker1 # [ 62.585571] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:370304237worker1 # [ 62.593975] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:370304238client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.15 seconds)4239client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4240worker2 # [ 62.674253] nix-grpc-daemon[790]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:526424241client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4242client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4243client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4244client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4245worker2 # [ 62.704818] nix-grpc-daemon[790]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:526424246worker2 # [ 62.713825] nix-grpc-daemon[790]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4247worker2 # [ 62.732754] nix-grpc-daemon[790]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4248worker2 # [ 62.748609] nix-grpc-daemon[790]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4249client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds)4250client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4251client # finding garbage collector roots...4252client # removing stale temporary roots file "/nix/var/nix/temproots/939"4253client # removing stale temporary roots file "/nix/var/nix/temproots/883"4254client # removing stale temporary roots file "/nix/var/nix/temproots/862"4255client # removing stale temporary roots file "/nix/var/nix/temproots/847"4256client # removing stale temporary roots file "/nix/var/nix/temproots/916"4257client # removing stale temporary roots file "/nix/var/nix/temproots/895"4258client # removing stale temporary roots file "/nix/var/nix/temproots/924"4259client # removing stale temporary roots file "/nix/var/nix/temproots/903"4260client # deleting specified paths...4261client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4262client # deleting unused links...4263client # note: hard linking is currently saving 0.0 KiB4264worker1 # [ 63.178611] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:469344265worker1 # [ 63.187130] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344266worker1 # [ 63.198858] nix-grpc-daemon[902]: time=2026-09-23T10:35:16.841Z level=INFO msg="All 1 paths already cached"4267worker2 # [ 63.138869] nix-grpc-daemon[790]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413744268worker1 # [ 63.205645] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:469344269client # copying 3 paths...4270worker2 # [ 63.156164] nix-grpc-daemon[790]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413744271client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4272worker1 # [ 63.244686] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:469344273worker1 # [ 63.252169] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_out=2324274worker2 # [ 63.195216] nix-grpc-daemon[790]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413744275worker2 # [ 63.202888] nix-grpc-daemon[790]: ts=2026-09-23T10:35:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_out=1764276worker1 # [ 63.270724] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:469344277worker1 # [ 63.277519] nix-grpc-daemon[486]: ts=2026-09-23T10:35:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_out=1284278client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4279client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4280client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.46 seconds)4281(finished: subtest: outputs read back through any worker carry the cache signature, in 6.42 seconds)4282subtest: two clients wanting the same drv build it once4283client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4284client # [ 63.293311] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4285client # Running as unit: dedup1.service; invocation ID: ed827b1365664509be8d83113825609a4286client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.05 seconds)4287client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4288worker2 # [ 63.468379] nix-grpc-daemon[790]: ts=2026-09-23T10:35:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413744289client # [ 63.473451] nix[992]: don't know how to build these paths:4290client # [ 63.475157] nix[992]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4291worker1 # [ 63.548295] nix-grpc-daemon[486]: ts=2026-09-23T10:35:17Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04292worker1 # [ 63.554643] nix-grpc-daemon[486]: ts=2026-09-23T10:35:17Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14293worker2 # [ 63.494616] nix-grpc-daemon[790]: ts=2026-09-23T10:35:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744294worker2 # [ 63.509094] nix-grpc-daemon[790]: ts=2026-09-23T10:35:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413744295client # [ 63.520093] nix[992]: copying 1 paths to 'lb:50051'...4296client # [ 63.521343] nix[992]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4297worker2 # [ 63.522699] nix-grpc-daemon[790]: ts=2026-09-23T10:35:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413744298worker2 # [ 63.537836] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4299worker2 # [ 63.699241] nix-daemon[849]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)4300worker2 # [ 63.719126] nix-daemon[849]: reaped child process 1013, status = succeeded4301lb # [ 63.723977] niks3-server[977]: 2026/09/23 10:35:17 INFO Received uploads request method=POST path=/api/pending_closures4302worker2 # [ 63.733769] nix-grpc-daemon[861]: time=2026-09-23T10:35:17.381Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4303worker2 # [ 63.735294] nix-grpc-daemon[861]: time=2026-09-23T10:35:17.381Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4304lb # [ 63.752582] niks3-server[977]: 2026/09/23 10:35:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4305worker2 # [ 63.760777] nix-grpc-daemon[861]: time=2026-09-23T10:35:17.408Z level=INFO msg="Uploading 1 narinfos"4306lb # [ 63.764383] niks3-server[977]: 2026/09/23 10:35:17 INFO Signed narinfos id=15 count=14307lb # [ 63.773080] niks3-server[977]: 2026/09/23 10:35:17 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4308lb # [ 63.786275] niks3-server[977]: 2026/09/23 10:35:17 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4309lb # [ 63.799118] niks3-server[977]: 2026/09/23 10:35:17 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4310lb # [ 63.807581] niks3-server[977]: 2026/09/23 10:35:17 INFO Completed upload id=154311worker2 # [ 63.805909] nix-grpc-daemon[861]: time=2026-09-23T10:35:17.454Z level=INFO msg="Upload complete. (251ms)"4312lb # [ 63.810295] niks3-server[977]: 2026/09/23 10:35:17 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4313worker2 # [ 63.820144] nix-grpc-daemon[790]: ts=2026-09-23T10:35:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_in=4324314worker2 # [ 63.834723] nix-grpc-daemon[790]: ts=2026-09-23T10:35:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744315worker2 # [ 63.843887] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4316worker2 # [ 63.849734] nix-daemon[849]: reaped child process 1008, status = succeeded4317client # [ 63.856711] nix[992]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4318worker2 # [ 63.863005] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4319client # [ 64.377953] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4320client # Running as unit: dedup2.service; invocation ID: d7576999f73f4e39b8e954e0170c8d274321client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.10 seconds)4322worker1 # [ 64.675538] nix-grpc-daemon[486]: ts=2026-09-23T10:35:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:469344323client # [ 64.620489] nix[1008]: don't know how to build these paths:4324client # [ 64.621520] nix[1008]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4325worker1 # [ 64.696239] nix-grpc-daemon[486]: ts=2026-09-23T10:35:18Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04326worker2 # [ 64.642749] nix-grpc-daemon[790]: ts=2026-09-23T10:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744327client # [ 64.645803] nix[1008]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4328client: must succeed: sleep 24329client: (finished: must succeed: sleep 2, in 2.05 seconds)4330client: waiting for success: ! systemctl is-active dedup1 dedup24331worker2 # [ 68.125943] nix-daemon[849]: reaped child process 1017, status = succeeded4332worker2 # [ 68.229695] nix-daemon[849]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4333worker2 # [ 68.247862] nix-daemon[849]: reaped child process 1050, status = succeeded4334lb # [ 68.254607] niks3-server[977]: 2026/09/23 10:35:21 INFO Received uploads request method=POST path=/api/pending_closures4335worker2 # [ 68.267827] nix-grpc-daemon[861]: time=2026-09-23T10:35:21.912Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4336worker2 # [ 68.269638] nix-grpc-daemon[861]: time=2026-09-23T10:35:21.912Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4337lb # [ 68.300532] niks3-server[977]: 2026/09/23 10:35:21 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4338worker2 # [ 68.309738] nix-grpc-daemon[861]: time=2026-09-23T10:35:21.957Z level=INFO msg="Uploading 1 narinfos"4339lb # [ 68.311703] niks3-server[977]: 2026/09/23 10:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4340lb # [ 68.324697] niks3-server[977]: 2026/09/23 10:35:21 INFO Signed narinfos id=16 count=14341lb # [ 68.329983] niks3-server[977]: 2026/09/23 10:35:21 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4342lb # [ 68.343284] niks3-server[977]: 2026/09/23 10:35:21 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4343lb # [ 68.349684] niks3-server[977]: 2026/09/23 10:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4344lb # [ 68.354312] niks3-server[977]: 2026/09/23 10:35:21 INFO Completed upload id=164345worker2 # [ 68.348496] nix-grpc-daemon[861]: time=2026-09-23T10:35:21.996Z level=INFO msg="Upload complete. (271ms)"4346lb # [ 68.359270] niks3-server[977]: 2026/09/23 10:35:22 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4347lb # [ 68.363931] envoy[895]: conn peer=[2001:db8:1::1]:40056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9528 tx=4388 ms=51334348worker2 # [ 68.361326] nix-grpc-daemon[790]: ts=2026-09-23T10:35:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14349lb # [ 68.370185] envoy[895]: conn peer=[2001:db8:1::1]:47038 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=634350lb # [ 68.374349] envoy[895]: conn peer=[2001:db8:1::1]:47034 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1284351lb # [ 68.379191] envoy[895]: conn peer=[2001:db8:1::1]:47052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=364352lb # [ 68.382756] envoy[895]: conn peer=[2001:db8:1::1]:47044 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=514353worker2 # [ 68.378573] nix-grpc-daemon[790]: ts=2026-09-23T10:35:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14354worker2 # [ 68.386382] nix-daemon[849]: reaped child process 1015, status = succeeded4355client # [ 68.426658] systemd[1]: dedup1.service: Deactivated successfully.4356client # [ 68.430042] systemd[1]: dedup1.service: Consumed 159ms CPU time over 5.137s wall clock time, 41.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4357client # [ 68.437312] systemd[1]: dedup2.service: Deactivated successfully.4358client # [ 68.441108] systemd[1]: dedup2.service: Consumed 212ms CPU time over 4.059s wall clock time, 35.2M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4359client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4360client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4361client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4362worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4363worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4364worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4365worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4366worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4367worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4368worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4369worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4370(finished: subtest: two clients wanting the same drv build it once, in 6.17 seconds)4371subtest: --timeout and --max-silent-time reach the builder4372worker1 # [ 69.708880] nix-grpc-daemon[486]: ts=2026-09-23T10:35:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:469344373worker1 # [ 69.724272] nix-grpc-daemon[486]: ts=2026-09-23T10:35:23Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04374worker1 # [ 69.730441] nix-grpc-daemon[486]: ts=2026-09-23T10:35:23Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14375worker1 # [ 69.740163] nix-grpc-daemon[486]: ts=2026-09-23T10:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344376worker1 # [ 69.750483] nix-grpc-daemon[486]: ts=2026-09-23T10:35:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344377worker1 # [ 69.764362] nix-grpc-daemon[486]: ts=2026-09-23T10:35:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:469344378worker1 # [ 69.779100] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4379worker1 # [ 69.938224] nix-daemon[851]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4380worker1 # [ 69.957281] nix-daemon[851]: reaped child process 1038, status = succeeded4381lb # [ 69.901103] niks3-server[977]: 2026/09/23 10:35:23 INFO Received uploads request method=POST path=/api/pending_closures4382worker1 # [ 69.973105] nix-grpc-daemon[902]: time=2026-09-23T10:35:23.617Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4383worker1 # [ 69.974646] nix-grpc-daemon[902]: time=2026-09-23T10:35:23.617Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4384worker1 # [ 70.006212] nix-grpc-daemon[902]: time=2026-09-23T10:35:23.651Z level=INFO msg="Uploading 1 narinfos"4385lb # [ 69.936563] niks3-server[977]: 2026/09/23 10:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4386lb # [ 69.949210] niks3-server[977]: 2026/09/23 10:35:23 INFO Signed narinfos id=17 count=14387lb # [ 69.958041] niks3-server[977]: 2026/09/23 10:35:23 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4388lb # [ 69.967391] niks3-server[977]: 2026/09/23 10:35:23 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4389lb # [ 69.981535] niks3-server[977]: 2026/09/23 10:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4390lb # [ 69.987248] niks3-server[977]: 2026/09/23 10:35:23 INFO Completed upload id=174391worker1 # [ 70.049205] nix-grpc-daemon[902]: time=2026-09-23T10:35:23.694Z level=INFO msg="Upload complete. (252ms)"4392lb # [ 69.990834] niks3-server[977]: 2026/09/23 10:35:23 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4393worker1 # [ 70.061133] nix-grpc-daemon[486]: ts=2026-09-23T10:35:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_in=4244394worker1 # [ 70.076307] nix-grpc-daemon[486]: ts=2026-09-23T10:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344395worker1 # [ 70.088297] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4396worker1 # [ 70.093653] nix-daemon[851]: reaped child process 1033, status = succeeded4397worker1 # [ 70.107802] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4398worker1 # [ 76.126078] nix-grpc-daemon[486]: ts=2026-09-23T10:35:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04399worker1 # [ 76.144187] nix-daemon[851]: reaped child process 1040, status = succeeded4400worker1 # [ 76.145358] nix-daemon[851]: reaped child process 1042, status = succeeded4401worker1 # [ 76.351577] nix-grpc-daemon[486]: ts=2026-09-23T10:35:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:469344402worker1 # [ 76.367552] nix-grpc-daemon[486]: ts=2026-09-23T10:35:30Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04403worker2 # [ 76.310039] nix-grpc-daemon[790]: ts=2026-09-23T10:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744404worker1 # [ 76.373826] nix-grpc-daemon[486]: ts=2026-09-23T10:35:30Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14405worker2 # [ 76.326095] nix-grpc-daemon[790]: ts=2026-09-23T10:35:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413744406worker2 # [ 76.342091] nix-grpc-daemon[790]: ts=2026-09-23T10:35:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413744407worker2 # [ 76.356935] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4408worker2 # [ 76.521643] nix-daemon[849]: accepted connection from pid 1063, user nix-grpc-daemon (trusted)4409worker2 # [ 76.542494] nix-daemon[849]: reaped child process 1066, status = succeeded4410lb # [ 76.546524] niks3-server[977]: 2026/09/23 10:35:30 INFO Received uploads request method=POST path=/api/pending_closures4411worker2 # [ 76.556377] nix-grpc-daemon[861]: time=2026-09-23T10:35:30.204Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4412worker2 # [ 76.557946] nix-grpc-daemon[861]: time=2026-09-23T10:35:30.204Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4413lb # [ 76.588527] niks3-server[977]: 2026/09/23 10:35:30 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4414worker2 # [ 76.601368] nix-grpc-daemon[861]: time=2026-09-23T10:35:30.249Z level=INFO msg="Uploading 1 narinfos"4415lb # [ 76.602391] niks3-server[977]: 2026/09/23 10:35:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4416lb # [ 76.610937] niks3-server[977]: 2026/09/23 10:35:30 INFO Signed narinfos id=18 count=14417lb # [ 76.616219] niks3-server[977]: 2026/09/23 10:35:30 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4418worker2 # [ 76.626411] nix-grpc-daemon[861]: time=2026-09-23T10:35:30.274Z level=INFO msg="Upload complete. (252ms)"4419lb # [ 76.627415] niks3-server[977]: 2026/09/23 10:35:30 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4420lb # [ 76.631373] niks3-server[977]: 2026/09/23 10:35:30 INFO Completed upload id=184421lb # [ 76.634946] niks3-server[977]: 2026/09/23 10:35:30 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4422worker2 # [ 76.636336] nix-grpc-daemon[790]: ts=2026-09-23T10:35:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_in=4244423worker2 # [ 76.655379] nix-grpc-daemon[790]: ts=2026-09-23T10:35:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744424worker2 # [ 76.664996] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4425worker2 # [ 76.668192] nix-daemon[849]: reaped child process 1061, status = succeeded4426worker2 # [ 76.681055] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4427lb # [ 78.360571] envoy[895]: conn peer=[2001:db8:1::1]:47054 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1541 ms=49154428lb # [ 78.365687] envoy[895]: conn peer=[2001:db8:1::1]:47058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=37784429lb # [ 78.370563] envoy[895]: conn peer=[2001:db8:1::1]:47062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3027 tx=952 ms=64444430worker2 # [ 83.095573] nix-grpc-daemon[790]: ts=2026-09-23T10:35:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04431worker2 # [ 83.119288] nix-daemon[849]: reaped child process 1068, status = succeeded4432worker2 # [ 83.121296] nix-daemon[849]: reaped child process 1070, status = succeeded4433(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.85 seconds)4434subtest: the first client leaving does not fail the second4435client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4436client # [ 83.340830] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4437client # Running as unit: share1.service; invocation ID: 0510dfd86dbf472c8c699ae398c9ee6b4438client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.08 seconds)4439client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4440worker2 # [ 83.545262] nix-grpc-daemon[790]: ts=2026-09-23T10:35:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413744441client # [ 83.551093] nix[1082]: don't know how to build these paths:4442client # [ 83.552874] nix[1082]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4443worker1 # [ 83.629899] nix-grpc-daemon[486]: ts=2026-09-23T10:35:37Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04444worker1 # [ 83.636230] nix-grpc-daemon[486]: ts=2026-09-23T10:35:37Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14445worker1 # [ 83.643137] nix-grpc-daemon[486]: ts=2026-09-23T10:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344446worker1 # [ 83.654530] nix-grpc-daemon[486]: ts=2026-09-23T10:35:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344447client # [ 83.603579] nix[1082]: copying 1 paths to 'lb:50051'...4448client # [ 83.605349] nix[1082]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4449worker1 # [ 83.668583] nix-grpc-daemon[486]: ts=2026-09-23T10:35:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:469344450worker1 # [ 83.683342] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4451worker1 # [ 83.838259] nix-daemon[851]: accepted connection from pid 1069, user nix-grpc-daemon (trusted)4452worker1 # [ 83.859326] nix-daemon[851]: reaped child process 1072, status = succeeded4453lb # [ 83.801620] niks3-server[977]: 2026/09/23 10:35:37 INFO Received uploads request method=POST path=/api/pending_closures4454worker1 # [ 83.873813] nix-grpc-daemon[902]: time=2026-09-23T10:35:37.518Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4455worker1 # [ 83.875320] nix-grpc-daemon[902]: time=2026-09-23T10:35:37.518Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4456lb # [ 83.832427] niks3-server[977]: 2026/09/23 10:35:37 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4457worker1 # [ 83.917878] nix-grpc-daemon[902]: time=2026-09-23T10:35:37.562Z level=INFO msg="Uploading 1 narinfos"4458lb # [ 83.849197] niks3-server[977]: 2026/09/23 10:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4459lb # [ 83.858820] niks3-server[977]: 2026/09/23 10:35:37 INFO Signed narinfos id=19 count=14460lb # [ 83.867698] niks3-server[977]: 2026/09/23 10:35:37 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4461lb # [ 83.886432] niks3-server[977]: 2026/09/23 10:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4462worker1 # [ 83.956392] nix-grpc-daemon[902]: time=2026-09-23T10:35:37.601Z level=INFO msg="Upload complete. (256ms)"4463lb # [ 83.895670] niks3-server[977]: 2026/09/23 10:35:37 INFO Completed upload id=194464lb # [ 83.902279] niks3-server[977]: 2026/09/23 10:35:37 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4465worker1 # [ 83.966785] nix-grpc-daemon[486]: ts=2026-09-23T10:35:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_in=4324466worker1 # [ 83.980612] nix-grpc-daemon[486]: ts=2026-09-23T10:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344467worker1 # [ 83.994183] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4468worker1 # [ 83.999592] nix-daemon[851]: reaped child process 1067, status = succeeded4469worker1 # [ 84.011562] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4470client # [ 83.946062] nix[1082]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4471client # [ 84.439590] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4472client # Running as unit: share2.service; invocation ID: 987143722a5542e7a9b421171ddcbe004473client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4474worker2 # [ 84.651589] nix-grpc-daemon[790]: ts=2026-09-23T10:35:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413744475client # [ 84.656903] nix[1098]: don't know how to build these paths:4476client # [ 84.658457] nix[1098]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4477worker1 # [ 84.733398] nix-grpc-daemon[486]: ts=2026-09-23T10:35:38Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04478client # [ 84.676915] nix[1098]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4479worker1 # [ 84.741067] nix-grpc-daemon[486]: ts=2026-09-23T10:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344480client: must succeed: sleep 2; systemctl kill -s INT share14481client # [ 87.809343] systemd[1]: share1.service: Sent signal SIGINT to main process 1082 (nix) on client request.4482client # [ 87.820332] nix[1082]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4483client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4484client: waiting for success: ! systemctl is-active share14485client # [ 87.835973] nix[1082]: error: interrupted by the user4486client # [ 87.885278] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4487client # [ 87.887224] systemd[1]: share1.service: Failed with result 'exit-code'.4488client # [ 87.891098] systemd[1]: share1.service: Consumed 175ms CPU time over 4.545s wall clock time, 43.5M memory peak, 6.1K incoming IP traffic, 8K outgoing IP traffic.4489client: (finished: waiting for success: ! systemctl is-active share1, in 0.07 seconds)4490client: waiting for success: ! systemctl is-active share24491worker1 # [ 88.256387] nix-daemon[851]: reaped child process 1076, status = succeeded4492worker1 # [ 88.375018] nix-daemon[851]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)4493worker1 # [ 88.391608] nix-daemon[851]: reaped child process 1115, status = succeeded4494lb # [ 88.335140] niks3-server[977]: 2026/09/23 10:35:41 INFO Received uploads request method=POST path=/api/pending_closures4495worker1 # [ 88.410278] nix-grpc-daemon[902]: time=2026-09-23T10:35:42.053Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4496worker1 # [ 88.411768] nix-grpc-daemon[902]: time=2026-09-23T10:35:42.053Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4497lb # [ 88.360989] envoy[895]: conn peer=[2001:db8:1::1]:58124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3015 tx=977 ms=68244498lb # [ 88.371525] envoy[895]: conn peer=[2001:db8:1::1]:44536 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=771 ms=42974499worker1 # [ 88.451695] nix-grpc-daemon[902]: time=2026-09-23T10:35:42.096Z level=INFO msg="Uploading 1 narinfos"4500lb # [ 88.388732] niks3-server[977]: 2026/09/23 10:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4501lb # [ 88.400481] niks3-server[977]: 2026/09/23 10:35:42 INFO Signed narinfos id=20 count=14502lb # [ 88.409191] niks3-server[977]: 2026/09/23 10:35:42 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4503lb # [ 88.417743] niks3-server[977]: 2026/09/23 10:35:42 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4504worker1 # [ 88.491641] nix-grpc-daemon[902]: time=2026-09-23T10:35:42.136Z level=INFO msg="Upload complete. (263ms)"4505lb # [ 88.430724] niks3-server[977]: 2026/09/23 10:35:42 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4506lb # [ 88.436596] niks3-server[977]: 2026/09/23 10:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4507lb # [ 88.440722] niks3-server[977]: 2026/09/23 10:35:42 INFO Completed upload id=204508lb # [ 88.443908] niks3-server[977]: 2026/09/23 10:35:42 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4509worker1 # [ 88.505042] nix-grpc-daemon[486]: ts=2026-09-23T10:35:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14510worker1 # [ 88.524390] nix-grpc-daemon[486]: ts=2026-09-23T10:35:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=3 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14511worker1 # [ 88.531877] nix-daemon[851]: reaped child process 1074, status = succeeded4512client # [ 88.476541] systemd[1]: share2.service: Deactivated successfully.4513client # [ 88.481306] systemd[1]: share2.service: Consumed 184ms CPU time over 4.041s wall clock time, 35.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4514client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4515client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4516client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4517(finished: subtest: the first client leaving does not fail the second, in 6.05 seconds)4518subtest: build hook: nix-daemon with builders = grpc://lb4519client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4520client # [ 89.463541] systemd[1]: Started Nix Daemon.4521client # [ 89.587555] nix-daemon[1145]: accepted connection from pid 1142, user root (trusted)4522worker2 # [ 89.743915] nix-grpc-daemon[790]: ts=2026-09-23T10:35:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:413744523worker1 # [ 89.810080] nix-grpc-daemon[486]: ts=2026-09-23T10:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344524worker1 # [ 89.829887] nix-grpc-daemon[486]: ts=2026-09-23T10:35:43Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04525worker1 # [ 89.836360] nix-grpc-daemon[486]: ts=2026-09-23T10:35:43Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14526worker2 # [ 89.777167] nix-grpc-daemon[790]: ts=2026-09-23T10:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744527worker2 # [ 89.792544] nix-grpc-daemon[790]: ts=2026-09-23T10:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413744528worker2 # [ 89.807114] nix-grpc-daemon[790]: ts=2026-09-23T10:35:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413744529worker2 # [ 89.823215] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4530worker2 # [ 89.986403] nix-daemon[849]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4531worker2 # [ 90.005857] nix-daemon[849]: reaped child process 1112, status = succeeded4532lb # [ 90.011624] niks3-server[977]: 2026/09/23 10:35:43 INFO Received uploads request method=POST path=/api/pending_closures4533worker2 # [ 90.021202] nix-grpc-daemon[861]: time=2026-09-23T10:35:43.668Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4534worker2 # [ 90.022779] nix-grpc-daemon[861]: time=2026-09-23T10:35:43.668Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4535lb # [ 90.045707] niks3-server[977]: 2026/09/23 10:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4536worker2 # [ 90.055390] nix-grpc-daemon[861]: time=2026-09-23T10:35:43.703Z level=INFO msg="Uploading 1 narinfos"4537lb # [ 90.059913] niks3-server[977]: 2026/09/23 10:35:43 INFO Signed narinfos id=21 count=14538lb # [ 90.068215] niks3-server[977]: 2026/09/23 10:35:43 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4539lb # [ 90.076297] niks3-server[977]: 2026/09/23 10:35:43 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4540lb # [ 90.084107] niks3-server[977]: 2026/09/23 10:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4541lb # [ 90.094838] niks3-server[977]: 2026/09/23 10:35:43 INFO Completed upload id=214542worker2 # [ 90.094212] nix-grpc-daemon[861]: time=2026-09-23T10:35:43.742Z level=INFO msg="Upload complete. (252ms)"4543lb # [ 90.100720] niks3-server[977]: 2026/09/23 10:35:43 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4544worker2 # [ 90.104430] nix-grpc-daemon[790]: ts=2026-09-23T10:35:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_in=5124545worker2 # [ 90.124929] nix-grpc-daemon[790]: ts=2026-09-23T10:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744546worker2 # [ 90.132645] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4547worker2 # [ 90.138581] nix-daemon[849]: reaped child process 1107, status = succeeded4548worker2 # [ 90.151075] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4549worker2 # [ 91.422789] nix-daemon[849]: reaped child process 1116, status = succeeded4550worker2 # [ 91.539215] nix-daemon[849]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)4551worker2 # [ 91.558595] nix-daemon[849]: reaped child process 1135, status = succeeded4552lb # [ 91.563734] niks3-server[977]: 2026/09/23 10:35:45 INFO Received uploads request method=POST path=/api/pending_closures4553worker2 # [ 91.574655] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.221Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4554worker2 # [ 91.576459] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.221Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4555lb # [ 91.605199] niks3-server[977]: 2026/09/23 10:35:45 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4556worker2 # [ 91.619034] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.266Z level=INFO msg="Uploading 1 narinfos"4557lb # [ 91.622527] niks3-server[977]: 2026/09/23 10:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4558lb # [ 91.630246] niks3-server[977]: 2026/09/23 10:35:45 INFO Signed narinfos id=22 count=14559lb # [ 91.636105] niks3-server[977]: 2026/09/23 10:35:45 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4560lb # [ 91.641465] niks3-server[977]: 2026/09/23 10:35:45 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4561lb # [ 91.648694] niks3-server[977]: 2026/09/23 10:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4562lb # [ 91.653676] niks3-server[977]: 2026/09/23 10:35:45 INFO Completed upload id=224563worker2 # [ 91.653792] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.301Z level=INFO msg="Upload complete. (284ms)"4564lb # [ 91.657780] niks3-server[977]: 2026/09/23 10:35:45 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4565worker2 # [ 91.665130] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14566worker2 # [ 91.684596] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413744567worker2 # [ 91.690979] nix-daemon[849]: reaped child process 1114, status = succeeded4568worker2 # [ 91.712560] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413744569worker2 # [ 91.719414] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_out=1284570worker1 # [ 92.014025] nix-grpc-daemon[486]: ts=2026-09-23T10:35:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:469344571worker2 # [ 91.953639] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413744572worker2 # [ 91.964427] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.612Z level=INFO msg="All 1 paths already cached"4573worker1 # [ 92.045882] nix-grpc-daemon[486]: ts=2026-09-23T10:35:45Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04574worker1 # [ 92.052366] nix-grpc-daemon[486]: ts=2026-09-23T10:35:45Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14575worker2 # [ 91.992631] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744576worker2 # [ 92.007210] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413744577worker2 # [ 92.021090] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.669Z level=INFO msg="All 1 paths already cached"4578worker2 # [ 92.030401] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413744579worker2 # [ 92.047464] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4580worker2 # [ 92.200344] nix-daemon[849]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)4581worker2 # [ 92.221060] nix-daemon[849]: reaped child process 1142, status = succeeded4582lb # [ 92.219582] niks3-server[977]: 2026/09/23 10:35:45 INFO Received uploads request method=POST path=/api/pending_closures4583worker2 # [ 92.227869] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.876Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4584worker2 # [ 92.230031] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.877Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4585lb # [ 92.250632] niks3-server[977]: 2026/09/23 10:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4586worker2 # [ 92.256893] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.904Z level=INFO msg="Uploading 1 narinfos"4587lb # [ 92.264379] niks3-server[977]: 2026/09/23 10:35:45 INFO Signed narinfos id=23 count=14588lb # [ 92.267591] niks3-server[977]: 2026/09/23 10:35:45 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4589worker2 # [ 92.286431] nix-grpc-daemon[861]: time=2026-09-23T10:35:45.934Z level=INFO msg="Upload complete. (225ms)"4590lb # [ 92.286419] niks3-server[977]: 2026/09/23 10:35:45 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4591lb # [ 92.290030] niks3-server[977]: 2026/09/23 10:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4592worker2 # [ 92.292319] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_in=6244593lb # [ 92.297128] niks3-server[977]: 2026/09/23 10:35:45 INFO Completed upload id=234594lb # [ 92.299127] niks3-server[977]: 2026/09/23 10:35:45 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4595worker2 # [ 92.304319] nix-grpc-daemon[790]: ts=2026-09-23T10:35:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744596worker2 # [ 92.314911] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4597worker2 # [ 92.319464] nix-daemon[849]: reaped child process 1137, status = succeeded4598worker2 # [ 92.330593] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4599worker2 # [ 93.816517] nix-daemon[849]: reaped child process 1146, status = succeeded4600worker2 # [ 93.911131] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)4601worker2 # [ 93.930238] nix-daemon[849]: reaped child process 1165, status = succeeded4602lb # [ 93.937763] niks3-server[977]: 2026/09/23 10:35:47 INFO Received uploads request method=POST path=/api/pending_closures4603worker2 # [ 93.948401] nix-grpc-daemon[861]: time=2026-09-23T10:35:47.593Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4604worker2 # [ 93.949960] nix-grpc-daemon[861]: time=2026-09-23T10:35:47.593Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4605lb # [ 93.979296] niks3-server[977]: 2026/09/23 10:35:47 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4606lb # [ 93.994895] niks3-server[977]: 2026/09/23 10:35:47 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4607worker2 # [ 94.003634] nix-grpc-daemon[861]: time=2026-09-23T10:35:47.651Z level=INFO msg="Uploading 1 narinfos"4608lb # [ 94.003475] niks3-server[977]: 2026/09/23 10:35:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4609lb # [ 94.014265] niks3-server[977]: 2026/09/23 10:35:47 INFO Signed narinfos id=24 count=14610lb # [ 94.020359] niks3-server[977]: 2026/09/23 10:35:47 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4611lb # [ 94.032249] niks3-server[977]: 2026/09/23 10:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4612lb # [ 94.038039] niks3-server[977]: 2026/09/23 10:35:47 INFO Completed upload id=244613worker2 # [ 94.037054] nix-grpc-daemon[861]: time=2026-09-23T10:35:47.685Z level=INFO msg="Upload complete. (291ms)"4614lb # [ 94.040933] niks3-server[977]: 2026/09/23 10:35:47 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4615worker2 # [ 94.049183] nix-grpc-daemon[790]: ts=2026-09-23T10:35:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14616worker2 # [ 94.068453] nix-grpc-daemon[790]: ts=2026-09-23T10:35:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413744617worker2 # [ 94.074950] nix-daemon[849]: reaped child process 1144, status = succeeded4618worker2 # [ 94.096596] nix-grpc-daemon[790]: ts=2026-09-23T10:35:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413744619worker2 # [ 94.103727] nix-grpc-daemon[790]: ts=2026-09-23T10:35:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_out=1764620worker1 # [ 94.402361] nix-grpc-daemon[486]: ts=2026-09-23T10:35:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:469344621worker1 # [ 94.410327] nix-grpc-daemon[486]: ts=2026-09-23T10:35:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344622worker1 # [ 94.438736] nix-grpc-daemon[486]: ts=2026-09-23T10:35:48Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04623worker1 # [ 94.445254] nix-grpc-daemon[486]: ts=2026-09-23T10:35:48Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14624worker2 # [ 94.385897] nix-grpc-daemon[790]: ts=2026-09-23T10:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744625worker2 # [ 94.400779] nix-grpc-daemon[790]: ts=2026-09-23T10:35:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413744626worker2 # [ 94.414445] nix-grpc-daemon[861]: time=2026-09-23T10:35:48.062Z level=INFO msg="All 2 paths already cached"4627worker2 # [ 94.424656] nix-grpc-daemon[790]: ts=2026-09-23T10:35:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413744628worker2 # [ 94.438251] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4629worker2 # [ 94.608412] nix-daemon[849]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)4630worker2 # [ 94.626359] nix-daemon[849]: reaped child process 1172, status = succeeded4631lb # [ 94.630885] niks3-server[977]: 2026/09/23 10:35:48 INFO Received uploads request method=POST path=/api/pending_closures4632worker2 # [ 94.641445] nix-grpc-daemon[861]: time=2026-09-23T10:35:48.289Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4633worker2 # [ 94.643033] nix-grpc-daemon[861]: time=2026-09-23T10:35:48.289Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4634lb # [ 94.664997] niks3-server[977]: 2026/09/23 10:35:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4635worker2 # [ 94.673220] nix-grpc-daemon[861]: time=2026-09-23T10:35:48.321Z level=INFO msg="Uploading 1 narinfos"4636lb # [ 94.678705] niks3-server[977]: 2026/09/23 10:35:48 INFO Signed narinfos id=25 count=14637lb # [ 94.684249] niks3-server[977]: 2026/09/23 10:35:48 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4638lb # [ 94.693555] niks3-server[977]: 2026/09/23 10:35:48 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4639worker2 # [ 94.700101] nix-grpc-daemon[861]: time=2026-09-23T10:35:48.348Z level=INFO msg="Upload complete. (243ms)"4640lb # [ 94.701587] niks3-server[977]: 2026/09/23 10:35:48 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4641lb # [ 94.705130] niks3-server[977]: 2026/09/23 10:35:48 INFO Completed upload id=254642lb # [ 94.708166] niks3-server[977]: 2026/09/23 10:35:48 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4643worker2 # [ 94.708267] nix-grpc-daemon[790]: ts=2026-09-23T10:35:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_in=7524644worker2 # [ 94.723482] nix-grpc-daemon[790]: ts=2026-09-23T10:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744645worker2 # [ 94.732251] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4646worker2 # [ 94.737131] nix-daemon[849]: reaped child process 1167, status = succeeded4647worker2 # [ 94.751593] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4648worker2 # [ 96.024981] nix-daemon[849]: reaped child process 1176, status = succeeded4649worker2 # [ 96.138269] nix-daemon[849]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)4650worker2 # [ 96.153517] nix-daemon[849]: reaped child process 1195, status = succeeded4651lb # [ 96.161796] niks3-server[977]: 2026/09/23 10:35:49 INFO Received uploads request method=POST path=/api/pending_closures4652worker2 # [ 96.172366] nix-grpc-daemon[861]: time=2026-09-23T10:35:49.819Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4653worker2 # [ 96.174103] nix-grpc-daemon[861]: time=2026-09-23T10:35:49.819Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4654lb # [ 96.205029] niks3-server[977]: 2026/09/23 10:35:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4655worker2 # [ 96.213874] nix-grpc-daemon[861]: time=2026-09-23T10:35:49.861Z level=INFO msg="Uploading 1 narinfos"4656lb # [ 96.217087] niks3-server[977]: 2026/09/23 10:35:49 INFO Signed narinfos id=26 count=14657lb # [ 96.226537] niks3-server[977]: 2026/09/23 10:35:49 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4658lb # [ 96.234625] niks3-server[977]: 2026/09/23 10:35:49 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4659lb # [ 96.245085] niks3-server[977]: 2026/09/23 10:35:49 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4660lb # [ 96.252141] niks3-server[977]: 2026/09/23 10:35:49 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4661worker2 # [ 96.251221] nix-grpc-daemon[861]: time=2026-09-23T10:35:49.899Z level=INFO msg="Upload complete. (287ms)"4662lb # [ 96.255657] niks3-server[977]: 2026/09/23 10:35:49 INFO Completed upload id=264663lb # [ 96.258053] niks3-server[977]: 2026/09/23 10:35:49 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4664worker2 # [ 96.265161] nix-grpc-daemon[790]: ts=2026-09-23T10:35:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14665worker2 # [ 96.283270] nix-grpc-daemon[790]: ts=2026-09-23T10:35:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413744666worker2 # [ 96.290734] nix-daemon[849]: reaped child process 1174, status = succeeded4667worker2 # [ 96.313379] nix-grpc-daemon[790]: ts=2026-09-23T10:35:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413744668worker2 # [ 96.320139] nix-grpc-daemon[790]: ts=2026-09-23T10:35:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_out=2324669client # [ 96.402696] nix-daemon[1145]: reaped child process 1147, status = succeeded4670client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.10 seconds)4671client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4672client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4673client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4674client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4675client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4676client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4677(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.22 seconds)4678subtest: an input only one worker has still reaches the builder4679client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4680client # this derivation will be built:4681client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4682client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4683client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)4684client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4685client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.11 seconds)4686worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4687worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4688worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4689worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4690client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24691client # [ 97.354830] nix-daemon[1145]: accepted connection from pid 1227, user root (trusted)4692client # this derivation will be built:4693client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4694client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4695worker1 # [ 97.560703] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:469344696worker1 # [ 97.569365] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344697worker1 # [ 97.734061] nix-daemon[851]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)4698worker1 # [ 97.753938] nix-daemon[851]: reaped child process 1131, status = succeeded4699lb # [ 97.691151] niks3-server[977]: 2026/09/23 10:35:51 INFO Received uploads request method=POST path=/api/pending_closures4700worker1 # [ 97.764086] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.408Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4701worker1 # [ 97.765829] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.408Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4702lb # [ 97.724838] niks3-server[977]: 2026/09/23 10:35:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4703worker1 # [ 97.797523] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.441Z level=INFO msg="Uploading 1 narinfos"4704lb # [ 97.738484] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4705lb # [ 97.750537] niks3-server[977]: 2026/09/23 10:35:51 INFO Signed narinfos id=27 count=14706worker1 # [ 97.823821] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.468Z level=INFO msg="Upload complete. (255ms)"4707lb # [ 97.760454] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4708lb # [ 97.769160] niks3-server[977]: 2026/09/23 10:35:51 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4709lb # [ 97.773016] niks3-server[977]: 2026/09/23 10:35:51 INFO Completed upload id=274710lb # [ 97.776765] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4711worker1 # [ 97.846385] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04712worker1 # [ 97.854107] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14713worker1 # [ 97.863078] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344714worker1 # [ 97.873876] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344715client # copying 2 paths to 'lb:50051'...4716client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4717worker1 # [ 97.891360] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:469344718client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4719worker1 # [ 97.905428] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4720worker1 # [ 98.051782] nix-daemon[851]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)4721worker1 # [ 98.072498] nix-daemon[851]: reaped child process 1140, status = succeeded4722lb # [ 98.013148] niks3-server[977]: 2026/09/23 10:35:51 INFO Received uploads request method=POST path=/api/pending_closures4723lb # [ 98.022405] niks3-server[977]: 2026/09/23 10:35:51 INFO Received uploads request method=POST path=/api/pending_closures4724worker1 # [ 98.095430] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.738Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4725worker1 # [ 98.098306] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.738Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4726worker1 # [ 98.100992] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.738Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4727lb # [ 98.059265] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4728lb # [ 98.073512] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4729lb # [ 98.084957] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4730lb # [ 98.095683] niks3-server[977]: 2026/09/23 10:35:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4731lb # [ 98.103756] niks3-server[977]: 2026/09/23 10:35:51 INFO Signed narinfos id=28 count=14732worker1 # [ 98.165342] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.809Z level=INFO msg="Uploading 3 narinfos"4733lb # [ 98.108916] niks3-server[977]: 2026/09/23 10:35:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4734lb # [ 98.115652] niks3-server[977]: 2026/09/23 10:35:51 INFO Signed narinfos id=29 count=24735lb # [ 98.120708] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4736lb # [ 98.137777] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4737lb # [ 98.149536] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4738lb # [ 98.153194] niks3-server[977]: 2026/09/23 10:35:51 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4739worker1 # [ 98.215829] nix-grpc-daemon[902]: time=2026-09-23T10:35:51.860Z level=INFO msg="Upload complete. (294ms)"4740lb # [ 98.156178] niks3-server[977]: 2026/09/23 10:35:51 INFO Completed upload id=284741lb # [ 98.157954] niks3-server[977]: 2026/09/23 10:35:51 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4742lb # [ 98.161548] niks3-server[977]: 2026/09/23 10:35:51 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4743lb # [ 98.167356] niks3-server[977]: 2026/09/23 10:35:51 INFO Completed upload id=294744worker1 # [ 98.228767] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=2 nar_bytes_in=9844745worker1 # [ 98.246638] nix-grpc-daemon[486]: ts=2026-09-23T10:35:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344746worker1 # [ 98.259840] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4747worker1 # [ 98.264279] nix-daemon[851]: reaped child process 1135, status = succeeded4748client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4749client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4750worker1 # [ 98.284112] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4751lb # [ 98.359698] envoy[895]: conn peer=[2001:db8:1::1]:44542 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=38094752lb # [ 98.364713] envoy[895]: conn peer=[2001:db8:1::1]:41518 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=264753lb # [ 98.369540] envoy[895]: conn peer=[2001:db8:1::1]:44552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1944 ms=19944754lb # [ 98.375435] envoy[895]: conn peer=[2001:db8:1::1]:41544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=284755lb # [ 98.380145] envoy[895]: conn peer=[2001:db8:1::1]:41528 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3914 tx=2157 ms=21734756lb # [ 98.384846] envoy[895]: conn peer=[2001:db8:1::1]:41562 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=274757lb # [ 98.389510] envoy[895]: conn peer=[2001:db8:1::1]:41556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4365 tx=2415 ms=19994758client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4759worker1 # [ 99.551317] nix-daemon[851]: reaped child process 1144, status = succeeded4760worker1 # [ 99.663205] nix-daemon[851]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)4761worker1 # [ 99.685613] nix-daemon[851]: reaped child process 1163, status = succeeded4762lb # [ 99.622503] niks3-server[977]: 2026/09/23 10:35:53 INFO Received uploads request method=POST path=/api/pending_closures4763worker1 # [ 99.695200] nix-grpc-daemon[902]: time=2026-09-23T10:35:53.340Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4764lb # [ 99.654989] niks3-server[977]: 2026/09/23 10:35:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign4765worker1 # [ 99.727119] nix-grpc-daemon[902]: time=2026-09-23T10:35:53.372Z level=INFO msg="Uploading 1 narinfos"4766lb # [ 99.668798] niks3-server[977]: 2026/09/23 10:35:53 INFO Signed narinfos id=30 count=14767lb # [ 99.671847] niks3-server[977]: 2026/09/23 10:35:53 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4768worker1 # [ 99.750715] nix-grpc-daemon[902]: time=2026-09-23T10:35:53.395Z level=INFO msg="Upload complete. (253ms)"4769lb # [ 99.689356] niks3-server[977]: 2026/09/23 10:35:53 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4770lb # [ 99.695431] niks3-server[977]: 2026/09/23 10:35:53 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete4771lb # [ 99.699657] niks3-server[977]: 2026/09/23 10:35:53 INFO Completed upload id=304772worker1 # [ 99.761005] nix-grpc-daemon[486]: ts=2026-09-23T10:35:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=14773lb # [ 99.704822] niks3-server[977]: 2026/09/23 10:35:53 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4774client # copying 1 paths...4775worker1 # [ 99.779800] nix-grpc-daemon[486]: ts=2026-09-23T10:35:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:469344776worker1 # [ 99.784636] nix-daemon[851]: reaped child process 1142, status = succeeded4777client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4778worker1 # [ 99.811071] nix-grpc-daemon[486]: ts=2026-09-23T10:35:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:469344779worker1 # [ 99.817695] nix-grpc-daemon[486]: ts=2026-09-23T10:35:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_out=1204780client # [ 99.837924] nix-daemon[1145]: reaped child process 1230, status = succeeded4781client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.68 seconds)4782client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24783client # [ 100.010895] nix-daemon[1145]: accepted connection from pid 1249, user root (trusted)4784client # this derivation will be built:4785client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4786worker1 # [ 100.232127] nix-grpc-daemon[486]: ts=2026-09-23T10:35:53Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:469344787client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4788worker1 # [ 100.241267] nix-grpc-daemon[486]: ts=2026-09-23T10:35:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344789worker1 # [ 100.250923] nix-grpc-daemon[902]: time=2026-09-23T10:35:53.896Z level=INFO msg="All 1 paths already cached"4790worker1 # [ 100.270042] nix-grpc-daemon[486]: ts=2026-09-23T10:35:53Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04791worker1 # [ 100.277765] nix-grpc-daemon[486]: ts=2026-09-23T10:35:53Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14792worker2 # [ 100.219794] nix-grpc-daemon[790]: ts=2026-09-23T10:35:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744793worker2 # [ 100.235958] nix-grpc-daemon[790]: ts=2026-09-23T10:35:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413744794client # copying 1 paths to 'lb:50051'...4795client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4796worker2 # [ 100.256458] nix-grpc-daemon[790]: ts=2026-09-23T10:35:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413744797worker2 # [ 100.271342] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4798worker2 # [ 100.294516] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)4799worker2 # [ 100.369267] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 2/5)4800worker2 # [ 100.389923] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 386 ms (attempt 3/5)4801worker2 # [ 100.778129] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 105 ms (attempt 4/5)4802worker2 # [ 100.885263] nix-grpc-daemon[790]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4803worker2 # [ 100.891747] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4804worker2 # [ 100.899856] nix-grpc-daemon[790]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4805worker2 # [ 101.072219] nix-daemon[849]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)4806worker2 # [ 101.091111] nix-daemon[849]: reaped child process 1210, status = succeeded4807lb # [ 101.098598] niks3-server[977]: 2026/09/23 10:35:54 INFO Received uploads request method=POST path=/api/pending_closures4808worker2 # [ 101.108432] nix-grpc-daemon[861]: time=2026-09-23T10:35:54.755Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4809worker2 # [ 101.110792] nix-grpc-daemon[861]: time=2026-09-23T10:35:54.755Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4810lb # [ 101.128100] niks3-server[977]: 2026/09/23 10:35:54 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4811lb # [ 101.141553] niks3-server[977]: 2026/09/23 10:35:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign4812worker2 # [ 101.147165] nix-grpc-daemon[861]: time=2026-09-23T10:35:54.795Z level=INFO msg="Uploading 1 narinfos"4813lb # [ 101.153129] niks3-server[977]: 2026/09/23 10:35:54 INFO Signed narinfos id=31 count=14814lb # [ 101.158106] niks3-server[977]: 2026/09/23 10:35:54 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4815worker2 # [ 101.168374] nix-grpc-daemon[861]: time=2026-09-23T10:35:54.816Z level=INFO msg="Upload complete. (249ms)"4816lb # [ 101.168823] niks3-server[977]: 2026/09/23 10:35:54 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete4817lb # [ 101.176232] niks3-server[977]: 2026/09/23 10:35:54 INFO Completed upload id=314818lb # [ 101.180284] niks3-server[977]: 2026/09/23 10:35:54 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4819worker2 # [ 101.176134] nix-grpc-daemon[790]: ts=2026-09-23T10:35:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_in=5684820worker2 # [ 101.187474] nix-grpc-daemon[790]: ts=2026-09-23T10:35:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413744821worker2 # [ 101.198521] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4822worker2 # [ 101.204486] nix-daemon[849]: reaped child process 1200, status = succeeded4823worker2 # [ 101.218249] nix-grpc-daemon[790]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4824client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4825worker2 # [ 101.232411] nix-daemon[849]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4826client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4827client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4828worker2 # [ 102.308896] nix-daemon[849]: reaped child process 1215, status = succeeded4829worker2 # [ 102.415921] nix-daemon[849]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)4830worker2 # [ 102.435641] nix-daemon[849]: reaped child process 1234, status = succeeded4831lb # [ 102.435969] niks3-server[977]: 2026/09/23 10:35:56 INFO Received uploads request method=POST path=/api/pending_closures4832worker2 # [ 102.444709] nix-grpc-daemon[861]: time=2026-09-23T10:35:56.092Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4833lb # [ 102.470303] niks3-server[977]: 2026/09/23 10:35:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign4834worker2 # [ 102.476190] nix-grpc-daemon[861]: time=2026-09-23T10:35:56.124Z level=INFO msg="Uploading 1 narinfos"4835lb # [ 102.484799] niks3-server[977]: 2026/09/23 10:35:56 INFO Signed narinfos id=32 count=14836lb # [ 102.492105] niks3-server[977]: 2026/09/23 10:35:56 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4837lb # [ 102.500523] niks3-server[977]: 2026/09/23 10:35:56 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4838lb # [ 102.509820] niks3-server[977]: 2026/09/23 10:35:56 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete4839lb # [ 102.519448] niks3-server[977]: 2026/09/23 10:35:56 INFO Completed upload id=324840worker2 # [ 102.517786] nix-grpc-daemon[861]: time=2026-09-23T10:35:56.165Z level=INFO msg="Upload complete. (271ms)"4841lb # [ 102.523333] niks3-server[977]: 2026/09/23 10:35:56 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4842client # copying 1 paths...4843worker2 # [ 102.529119] nix-grpc-daemon[790]: ts=2026-09-23T10:35:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=14844worker2 # [ 102.550891] nix-grpc-daemon[790]: ts=2026-09-23T10:35:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413744845worker2 # [ 102.556711] nix-daemon[849]: reaped child process 1212, status = succeeded4846client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4847worker2 # [ 102.584146] nix-grpc-daemon[790]: ts=2026-09-23T10:35:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413744848worker2 # [ 102.590861] nix-grpc-daemon[790]: ts=2026-09-23T10:35:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_out=1204849client # [ 102.671756] nix-daemon[1145]: reaped child process 1252, status = succeeded4850client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.83 seconds)4851client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24852client # [ 102.837064] nix-daemon[1145]: accepted connection from pid 1271, user root (trusted)4853client # this derivation will be built:4854client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4855client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4856worker1 # [ 103.045583] nix-grpc-daemon[486]: ts=2026-09-23T10:35:56Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:469344857worker1 # [ 103.053103] nix-grpc-daemon[486]: ts=2026-09-23T10:35:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344858worker1 # [ 103.061222] nix-grpc-daemon[902]: time=2026-09-23T10:35:56.703Z level=INFO msg="All 1 paths already cached"4859worker1 # [ 103.072309] nix-grpc-daemon[486]: ts=2026-09-23T10:35:56Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04860worker1 # [ 103.081629] nix-grpc-daemon[486]: ts=2026-09-23T10:35:56Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14861worker1 # [ 103.090555] nix-grpc-daemon[486]: ts=2026-09-23T10:35:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344862worker1 # [ 103.097990] nix-grpc-daemon[486]: ts=2026-09-23T10:35:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:469344863worker1 # [ 103.108162] nix-grpc-daemon[902]: time=2026-09-23T10:35:56.753Z level=INFO msg="All 1 paths already cached"4864client # copying 1 paths to 'lb:50051'...4865client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4866worker1 # [ 103.118645] nix-grpc-daemon[486]: ts=2026-09-23T10:35:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:469344867worker1 # [ 103.135123] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4868worker1 # [ 103.282780] nix-daemon[851]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)4869worker1 # [ 103.302592] nix-daemon[851]: reaped child process 1174, status = succeeded4870lb # [ 103.247205] niks3-server[977]: 2026/09/23 10:35:56 INFO Received uploads request method=POST path=/api/pending_closures4871worker1 # [ 103.321289] nix-grpc-daemon[902]: time=2026-09-23T10:35:56.966Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4872worker1 # [ 103.322846] nix-grpc-daemon[902]: time=2026-09-23T10:35:56.966Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4873lb # [ 103.278058] niks3-server[977]: 2026/09/23 10:35:56 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4874worker1 # [ 103.348821] nix-grpc-daemon[902]: time=2026-09-23T10:35:56.993Z level=INFO msg="Uploading 1 narinfos"4875lb # [ 103.288344] niks3-server[977]: 2026/09/23 10:35:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign4876lb # [ 103.296008] niks3-server[977]: 2026/09/23 10:35:56 INFO Signed narinfos id=33 count=14877lb # [ 103.304689] niks3-server[977]: 2026/09/23 10:35:56 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4878worker1 # [ 103.374011] nix-grpc-daemon[902]: time=2026-09-23T10:35:57.019Z level=INFO msg="Upload complete. (225ms)"4879lb # [ 103.314566] niks3-server[977]: 2026/09/23 10:35:56 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete4880worker1 # [ 103.385020] nix-grpc-daemon[486]: ts=2026-09-23T10:35:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=0 paths=1 nar_bytes_in=5684881lb # [ 103.322501] niks3-server[977]: 2026/09/23 10:35:56 INFO Completed upload id=334882lb # [ 103.328998] niks3-server[977]: 2026/09/23 10:35:56 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4883worker1 # [ 103.392421] nix-grpc-daemon[486]: ts=2026-09-23T10:35:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:469344884worker1 # [ 103.401873] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4885worker1 # [ 103.406381] nix-daemon[851]: reaped child process 1169, status = succeeded4886client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4887worker1 # [ 103.418399] nix-daemon[851]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4888client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4889client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4890worker1 # [ 104.500699] nix-daemon[851]: reaped child process 1178, status = succeeded4891worker1 # [ 104.591954] nix-daemon[851]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)4892worker1 # [ 104.607777] nix-daemon[851]: reaped child process 1197, status = succeeded4893lb # [ 104.552802] niks3-server[977]: 2026/09/23 10:35:58 INFO Received uploads request method=POST path=/api/pending_closures4894worker1 # [ 104.626708] nix-grpc-daemon[902]: time=2026-09-23T10:35:58.269Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4895lb # [ 104.587204] niks3-server[977]: 2026/09/23 10:35:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign4896worker1 # [ 104.661248] nix-grpc-daemon[902]: time=2026-09-23T10:35:58.306Z level=INFO msg="Uploading 1 narinfos"4897lb # [ 104.601474] niks3-server[977]: 2026/09/23 10:35:58 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4898lb # [ 104.610865] niks3-server[977]: 2026/09/23 10:35:58 INFO Signed narinfos id=34 count=14899lb # [ 104.616167] niks3-server[977]: 2026/09/23 10:35:58 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4900lb # [ 104.623425] niks3-server[977]: 2026/09/23 10:35:58 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete4901worker1 # [ 104.690881] nix-grpc-daemon[902]: time=2026-09-23T10:35:58.335Z level=INFO msg="Upload complete. (262ms)"4902lb # [ 104.631047] niks3-server[977]: 2026/09/23 10:35:58 INFO Completed upload id=344903lb # [ 104.633281] niks3-server[977]: 2026/09/23 10:35:58 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4904client # copying 1 paths...4905worker1 # [ 104.708194] nix-grpc-daemon[486]: ts=2026-09-23T10:35:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46934 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=14906worker2 # [ 104.647303] nix-grpc-daemon[790]: ts=2026-09-23T10:35:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:413744907worker1 # [ 104.721546] nix-daemon[851]: reaped child process 1176, status = succeeded4908worker2 # [ 104.659900] nix-grpc-daemon[790]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4909client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4910worker2 # [ 104.703324] nix-grpc-daemon[790]: ts=2026-09-23T10:35:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:413744911worker2 # [ 104.710228] nix-grpc-daemon[790]: ts=2026-09-23T10:35:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41374 duration_s=0 paths=1 nar_bytes_out=1204912client # [ 104.788435] nix-daemon[1145]: reaped child process 1274, status = succeeded4913client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.12 seconds)4914(finished: subtest: an input only one worker has still reaches the builder, in 8.26 seconds)4915subtest: upload whose reference the worker lacks is completed from the cache4916client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4917client # this derivation will be built:4918client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4919client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4920client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.31 seconds)4921client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4922client # this derivation will be built:4923client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4924client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4925client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)4926client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4927client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds)4928worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4929worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)4930worker2 # [ 105.764581] systemd[1]: Stopping Nix worker-protocol over gRPC...4931worker2 # [ 105.987431] nix-grpc-daemon[790]: ts=2026-09-23T10:35:59Z level=info event=signal_exit4932worker2 # [ 107.965252] nix-grpc-daemon[790]: ts=2026-09-23T10:36:01Z level=info event=scheduler_restarting4933worker1 # [ 108.041761] nix-grpc-daemon[486]: ts=2026-09-23T10:36:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:577044934worker1 # [ 108.050470] nix-grpc-daemon[486]: ts=2026-09-23T10:36:01Z level=info event=worker_gone addr=192.168.1.4:500514935worker2 # [ 108.007513] nix-grpc-daemon[790]: ts=2026-09-23T10:36:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4936worker2 # [ 108.025540] nix-daemon[849]: reaped child process 851, status = succeeded4937lb # [ 108.357707] envoy[895]: conn peer=[2001:db8:1::1]:41574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3918 tx=2084 ms=22744938lb # [ 108.362711] envoy[895]: conn peer=[2001:db8:1::1]:41584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=284939lb # [ 108.370626] envoy[895]: conn peer=[2001:db8:1::1]:41590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2124 ms=24384940lb # [ 108.380991] envoy[895]: conn peer=[2001:db8:1::1]:42224 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=294941lb # [ 108.387151] envoy[895]: conn peer=[2001:db8:1::1]:42232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3724 tx=2083 ms=17444942lb # [ 108.394818] envoy[895]: conn peer=[2001:db8:1::1]:42234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=284943lb # [ 108.400514] envoy[895]: conn peer=[2001:db8:1::4]:52256 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8427 tx=1368 ms=739714944worker2 # [ 109.491678] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4945worker2 # [ 109.497093] systemd[1]: Stopped Nix worker-protocol over gRPC.4946worker2 # [ 109.505290] systemd[1]: nix-grpc-daemon.service: Consumed 3.494s CPU time over 1min 15.767s wall clock time, 65.5M memory peak, 185K incoming IP traffic, 252.5K outgoing IP traffic.4947lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux4948lb: must succeed: curl -sf localhost:9901/clusters4949lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4950lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.08 seconds)4951client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24952worker1 # [ 109.878005] nix-grpc-daemon[486]: ts=2026-09-23T10:36:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:469344953worker1 # [ 109.887007] nix-grpc-daemon[486]: ts=2026-09-23T10:36:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:469344954client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4955client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)4956worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4957worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)4958worker2 # [ 109.936635] systemd[1]: Starting scheduler-token.service...4959worker2 # [ 110.003258] systemd[1]: scheduler-token.service: Deactivated successfully.4960worker2 # [ 110.008375] systemd[1]: Finished scheduler-token.service.4961worker2 # [ 110.009805] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 69ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4962worker2 # [ 110.027590] systemd[1]: Starting Nix worker-protocol over gRPC...4963worker2 # [ 110.138154] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4964worker2 # [ 110.140373] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4965worker2 # [ 110.144264] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14966worker2 # [ 110.150716] nix-grpc-daemon[1259]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4967worker2 # [ 110.159686] nix-grpc-daemon[1259]: E0000 00:00:1790159763.796802 1259 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4968worker2 # [ 110.192130] nix-grpc-daemon[1259]: E0000 00:00:1790159763.798843 1259 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4969worker2 # [ 110.194433] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:03Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket4970worker2 # [ 110.199274] nix-grpc-daemon[1259]: E0000 00:00:1790159763.812231 1302 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4971worker2 # [ 110.202313] nix-grpc-daemon[1259]: E0000 00:00:1790159763.827857 1302 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4972worker2 # [ 110.204822] nix-grpc-daemon[1259]: E0000 00:00:1790159763.828010 1302 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4973worker2 # [ 110.210189] systemd[1]: Started Nix worker-protocol over gRPC.4974worker2 # [ 110.212977] nix-grpc-daemon[1259]: E0000 00:00:1790159763.846547 1302 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4975worker2 # [ 110.215360] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:03Z level=info event=scheduler_connected addr=lb:500514976worker1 # [ 110.277832] nix-grpc-daemon[486]: ts=2026-09-23T10:36:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:577044977worker1 # [ 110.296319] nix-grpc-daemon[486]: ts=2026-09-23T10:36:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04978worker1 # [ 110.317174] systemd[1]: Stopping Nix worker-protocol over gRPC...4979worker1 # [ 110.363822] nix-grpc-daemon[486]: ts=2026-09-23T10:36:04Z level=info event=signal_exit4980lb # [ 110.789314] niks3-server[977]: 2026/09/23 10:36:04 INFO lead: released remote=192.168.1.3:532904981worker1 # [ 111.854369] nix-grpc-daemon[486]: ts=2026-09-23T10:36:05Z level=info event=scheduler_restarting4982worker2 # [ 111.792171] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:05Z level=info event=scheduler_restarting addr=lb:500514983lb # [ 111.789170] envoy[895]: [2026-09-23 10:36:05.438][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}4984worker1 # [ 111.858267] nix-grpc-daemon[486]: ts=2026-09-23T10:36:05Z level=info event=scheduler_restarting addr=lb:500514985worker1 # [ 111.865063] nix-grpc-daemon[486]: ts=2026-09-23T10:36:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:577044986worker1 # [ 111.873840] nix-grpc-daemon[486]: ts=2026-09-23T10:36:05Z level=info event=worker_gone addr=192.168.1.4:500514987worker1 # [ 111.878778] nix-grpc-daemon[486]: ts=2026-09-23T10:36:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:577044988worker1 # [ 111.883531] nix-grpc-daemon[486]: ts=2026-09-23T10:36:05Z level=info event=worker_gone addr=192.168.1.3:500514989worker1 # [ 111.911801] nix-daemon[851]: reaped child process 853, status = succeeded4990worker2 # [ 111.992053] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:05Z level=info event=scheduler_connected addr=lb:500514991worker2 # [ 111.995888] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4992worker1 # [ 112.080595] nix-grpc-daemon[486]: ts=2026-09-23T10:36:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4993worker2 # [ 112.197072] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:05Z level=info event=scheduler_connected addr=lb:500514994worker2 # [ 112.202207] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4995worker2 # [ 112.403107] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_connected addr=lb:500514996worker2 # [ 112.408444] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4997worker2 # [ 112.609251] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_connected addr=lb:500514998worker2 # [ 112.614325] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4999worker2 # [ 112.816932] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_connected addr=lb:500515000worker2 # [ 112.820995] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5001worker2 # [ 113.017550] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_connected addr=lb:500515002worker2 # [ 113.022588] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5003worker2 # [ 113.223412] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_connected addr=lb:500515004worker2 # [ 113.228528] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5005worker2 # [ 113.429874] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_connected addr=lb:500515006worker2 # [ 113.435184] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5007worker1 # [ 113.508930] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5008worker1 # [ 113.510783] systemd[1]: Stopped Nix worker-protocol over gRPC.5009worker1 # [ 113.519280] systemd[1]: nix-grpc-daemon.service: Consumed 2.943s CPU time over 1min 43.521s wall clock time, 67M memory peak, 485.6K incoming IP traffic, 264K outgoing IP traffic.5010lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5011lb: must succeed: curl -sf localhost:9901/clusters5012lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5013lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.07 seconds)5014client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25015worker2 # [ 113.635959] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_connected addr=lb:500515016worker2 # [ 113.640936] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5017worker2 # [ 113.771906] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540405018worker2 # [ 113.787465] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5019worker2 # [ 113.812512] nix-grpc-daemon[1259]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)5020worker2 # [ 113.843337] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_connected addr=lb:500515021worker2 # [ 113.847529] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5022worker2 # [ 113.886933] nix-grpc-daemon[1259]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 2/5)5023worker2 # [ 113.947593] nix-grpc-daemon[1259]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 323 ms (attempt 3/5)5024worker2 # [ 114.044133] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_connected addr=lb:500515025worker2 # [ 114.049149] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5026worker2 # [ 114.249986] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_connected addr=lb:500515027worker2 # [ 114.254944] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5028worker2 # [ 114.272458] nix-grpc-daemon[1259]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 238 ms (attempt 4/5)5029worker2 # [ 114.455959] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_connected addr=lb:500515030worker2 # [ 114.460994] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5031worker2 # [ 114.515527] nix-grpc-daemon[1259]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5032worker2 # [ 114.520642] nix-grpc-daemon[1259]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5033client # copying 2 paths...5034client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5035worker2 # [ 114.544842] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540405036client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5037worker2 # [ 114.561790] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5038worker2 # [ 114.592741] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=niks3_push_started pid=13195039worker2 # [ 114.662566] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_connected addr=lb:500515040worker2 # [ 114.666097] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5041worker2 # [ 114.752953] nix-grpc-daemon[1319]: time=2026-09-23T10:36:08.400Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5042worker2 # [ 114.860201] nix-daemon[849]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5043worker2 # [ 114.865571] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_connected addr=lb:500515044worker2 # [ 114.877749] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045worker2 # [ 114.879754] nix-daemon[849]: reaped child process 1335, status = succeeded5046lb # [ 114.888101] niks3-server[977]: 2026/09/23 10:36:08 INFO Received uploads request method=POST path=/api/pending_closures5047lb # [ 114.899629] niks3-server[977]: 2026/09/23 10:36:08 INFO Received uploads request method=POST path=/api/pending_closures5048worker2 # [ 114.908454] nix-grpc-daemon[1319]: time=2026-09-23T10:36:08.556Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5049worker2 # [ 114.913521] nix-grpc-daemon[1319]: time=2026-09-23T10:36:08.558Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5050worker2 # [ 114.915328] nix-grpc-daemon[1319]: time=2026-09-23T10:36:08.558Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5051lb # [ 114.962719] niks3-server[977]: 2026/09/23 10:36:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5052worker2 # [ 114.975797] nix-grpc-daemon[1319]: time=2026-09-23T10:36:08.622Z level=INFO msg="Uploading 3 narinfos"5053lb # [ 114.977667] niks3-server[977]: 2026/09/23 10:36:08 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5054lb # [ 114.988717] niks3-server[977]: 2026/09/23 10:36:08 INFO Signed narinfos id=35 count=15055lb # [ 114.990486] niks3-server[977]: 2026/09/23 10:36:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5056lb # [ 115.000181] niks3-server[977]: 2026/09/23 10:36:08 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5057lb # [ 115.002970] niks3-server[977]: 2026/09/23 10:36:08 INFO Signed narinfos id=36 count=25058lb # [ 115.013340] niks3-server[977]: 2026/09/23 10:36:08 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5059lb # [ 115.021436] niks3-server[977]: 2026/09/23 10:36:08 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5060lb # [ 115.031477] niks3-server[977]: 2026/09/23 10:36:08 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5061lb # [ 115.039164] niks3-server[977]: 2026/09/23 10:36:08 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5062lb # [ 115.043604] niks3-server[977]: 2026/09/23 10:36:08 INFO Completed upload id=355063lb # [ 115.046971] niks3-server[977]: 2026/09/23 10:36:08 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5064lb # [ 115.049977] niks3-server[977]: 2026/09/23 10:36:08 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5065worker2 # [ 115.050134] nix-grpc-daemon[1319]: time=2026-09-23T10:36:08.698Z level=INFO msg="Upload complete. (293ms)"5066lb # [ 115.052895] niks3-server[977]: 2026/09/23 10:36:08 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5067lb # [ 115.055190] niks3-server[977]: 2026/09/23 10:36:08 INFO Completed upload id=365068worker2 # [ 115.060488] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54040 duration_s=0 paths=2 nar_bytes_in=3125069worker2 # [ 115.080284] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_connected addr=lb:500515070worker2 # [ 115.084467] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5071worker2 # [ 115.089379] nix-daemon[849]: reaped child process 1317, status = succeeded5072client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.55 seconds)5073worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5074worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5075lb # [ 115.161317] niks3-server[977]: 2026/09/23 10:36:08 INFO lead: acquired remote=192.168.1.4:584545076worker2 # [ 115.161557] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_take_over5077worker1 # [ 115.256802] systemd[1]: Starting Nix worker-protocol over gRPC...5078worker2 # [ 115.273859] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_connected addr=lb:500515079worker2 # [ 115.279372] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5080worker1 # [ 115.404397] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5081worker1 # [ 115.406991] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5082worker1 # [ 115.410814] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15083worker1 # [ 115.416057] nix-grpc-daemon[1220]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5084worker1 # [ 115.417519] nix-grpc-daemon[1220]: E0000 00:00:1790159769.060602 1220 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5085worker1 # [ 115.421836] nix-grpc-daemon[1220]: E0000 00:00:1790159769.066888 1220 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5086lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5087lb: must succeed: curl -sf localhost:9901/clusters5088worker1 # [ 115.465136] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5089worker1 # [ 115.467372] nix-grpc-daemon[1220]: E0000 00:00:1790159769.076010 1263 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5090worker1 # [ 115.470069] nix-grpc-daemon[1220]: E0000 00:00:1790159769.092533 1263 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5091worker1 # [ 115.472412] nix-grpc-daemon[1220]: E0000 00:00:1790159769.092727 1263 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5092worker1 # [ 115.474730] nix-grpc-daemon[1220]: E0000 00:00:1790159769.109544 1263 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5093worker1 # [ 115.480236] systemd[1]: Started Nix worker-protocol over gRPC.5094worker1 # [ 115.482790] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=scheduler_connected addr=lb:500515095worker1 # [ 115.484573] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5096lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5097worker2 # [ 115.480548] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:09Z level=info event=scheduler_connected addr=lb:500515098worker2 # [ 115.485933] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5099worker1 # [ 115.685418] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=scheduler_connected addr=lb:500515100worker1 # [ 115.690681] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5101worker2 # [ 115.686970] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:09Z level=info event=scheduler_connected addr=lb:500515102worker2 # [ 115.720202] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:09Z level=info event=oidc_jwks provider=mock keys=15103worker2 # [ 115.725761] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540505104worker2 # [ 115.731171] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05105worker1 # [ 115.891273] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:09Z level=info event=scheduler_connected addr=lb:500515106worker2 # [ 115.831280] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:540505107worker2 # [ 115.837150] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05108lb: must succeed: curl -sf localhost:9901/clusters5109lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5110lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.15 seconds)5111(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.74 seconds)5112subtest: a niks3 restart does not cost the leader its role5113lb: must succeed: curl -sf localhost:9901/clusters5114lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5115lb: must succeed: curl -sf localhost:9901/clusters5116lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5117worker1 # [ 116.729006] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:10Z level=info event=scheduler_lock_lost status=2005118worker2 # [ 116.667026] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:10Z level=info event=scheduler_lock_lost status=2005119lb # [ 116.665492] niks3-server[977]: 2026/09/23 10:36:10 INFO Shutdown signal received, draining in-flight requests timeout=10s5120lb # [ 116.665651] niks3-server[977]: 2026/09/23 10:36:10 INFO lead: released remote=192.168.1.4:584545121lb # [ 116.683604] systemd[1]: Stopping niks3 server...5122lb # [ 116.693980] systemd[1]: niks3.service: Deactivated successfully.5123lb # [ 116.696157] systemd[1]: Stopped niks3 server.5124lb # [ 116.696882] systemd[1]: niks3.service: Consumed 616ms CPU time over 1min 32.138s wall clock time, 27.4M memory peak, 92K incoming IP traffic, 133.9K outgoing IP traffic.5125lb # [ 116.706368] systemd[1]: Starting niks3 server...5126lb # [ 116.762514] niks3-server[1146]: 2026/09/23 10:36:10 goose: no migrations to run. current version: 202609200000005127lb # [ 116.767165] niks3-server[1146]: 2026/09/23 10:36:10 OK 1_commit_pending_closure.sql (6.15ms)5128lb # [ 116.770559] niks3-server[1146]: 2026/09/23 10:36:10 OK 2_object_stats_trigger.sql (3.05ms)5129lb # [ 116.772407] niks3-server[1146]: 2026/09/23 10:36:10 goose: up to current file version: 25130lb # [ 116.774206] niks3-server[1146]: 2026/09/23 10:36:10 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5131lb # [ 116.784063] niks3-server[1146]: 2026/09/23 10:36:10 INFO Read proxy enabled — serving cache objects from S35132lb # [ 116.788901] niks3-server[1146]: 2026/09/23 10:36:10 INFO Using socket-activated listener address=0.0.0.0:57515133lb # [ 116.790662] niks3-server[1146]: 2026/09/23 10:36:10 INFO systemd watchdog enabled interval=15s5134lb # [ 116.793031] niks3-server[1146]: 2026/09/23 10:36:10 INFO Starting HTTP server address=0.0.0.0:57515135lb # [ 116.794983] systemd[1]: Started niks3 server.5136lb: waiting for unit niks3.service5137lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5138lb: must succeed: curl -sf localhost:9901/clusters5139lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5140worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5141worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5142lb: must succeed: curl -sf localhost:9901/clusters5143lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5144lb: must succeed: curl -sf localhost:9901/clusters5145lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5146worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5147worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5148client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25149client # don't know how to build these paths:5150client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5151worker2 # [ 117.259676] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:540405152worker1 # [ 117.330794] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355725153worker1 # [ 117.346369] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5154worker1 # [ 117.372578] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5)5155worker1 # [ 117.472151] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 2/5)5156worker1 # [ 117.586059] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 305 ms (attempt 3/5)5157lb # [ 117.671525] niks3-server[1146]: 2026/09/23 10:36:11 INFO lead: acquired remote=192.168.1.4:409765158worker1 # [ 117.894004] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 4/5)5159worker1 # [ 117.907491] nix-grpc-daemon[1220]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5160worker1 # [ 117.913347] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5161worker2 # [ 117.889446] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:11Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05162worker2 # [ 117.897828] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:11Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15163worker2 # [ 117.905340] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540405164worker2 # [ 117.919996] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540405165client # copying 1 paths to 'lb:50051'...5166client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5167worker2 # [ 117.934207] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540405168worker2 # [ 117.949052] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5169worker2 # [ 118.109642] nix-daemon[849]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5170worker2 # [ 118.125409] nix-daemon[849]: reaped child process 1357, status = succeeded5171lb # [ 118.133420] niks3-server[1146]: 2026/09/23 10:36:11 INFO Received uploads request method=POST path=/api/pending_closures5172worker2 # [ 118.147931] nix-grpc-daemon[1319]: time=2026-09-23T10:36:11.795Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5173worker2 # [ 118.149877] nix-grpc-daemon[1319]: time=2026-09-23T10:36:11.795Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5174lb # [ 118.174463] niks3-server[1146]: 2026/09/23 10:36:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5175lb # [ 118.217958] niks3-server[1146]: 2026/09/23 10:36:11 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5176lb # [ 118.236769] niks3-server[1146]: 2026/09/23 10:36:11 INFO Signed narinfos id=37 count=15177lb # [ 118.239617] niks3-server[1146]: 2026/09/23 10:36:11 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5178worker2 # [ 118.241393] nix-grpc-daemon[1319]: time=2026-09-23T10:36:11.888Z level=INFO msg="Uploading 1 narinfos"5179lb # [ 118.249851] niks3-server[1146]: 2026/09/23 10:36:11 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5180lb # [ 118.263621] niks3-server[1146]: 2026/09/23 10:36:11 INFO Completed upload id=375181worker2 # [ 118.263820] nix-grpc-daemon[1319]: time=2026-09-23T10:36:11.911Z level=INFO msg="Upload complete. (298ms)"5182lb # [ 118.270433] niks3-server[1146]: 2026/09/23 10:36:11 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5183worker2 # [ 118.274979] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54040 duration_s=0 paths=1 nar_bytes_in=5445184worker2 # [ 118.288821] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540405185worker2 # [ 118.306193] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5186worker2 # [ 118.310108] nix-daemon[849]: reaped child process 1352, status = succeeded5187client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5188client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5189worker2 # [ 118.323637] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5190lb # [ 118.357580] envoy[895]: conn peer=[2001:db8:1::1]:42248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=295191lb # [ 118.362335] envoy[895]: conn peer=[2001:db8:1::3]:60466 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1960 tx=886 ms=749925192lb # [ 118.367432] envoy[895]: conn peer=[2001:db8:1::1]:40250 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1007 tx=305 ms=13045193client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5194client # farm-a> LOG-farm-a-after-niks3-restart5195worker2 # [ 119.389312] nix-daemon[849]: reaped child process 1361, status = succeeded5196worker2 # [ 119.518742] nix-daemon[849]: accepted connection from pid 1377, user nix-grpc-daemon (trusted)5197worker2 # [ 119.534389] nix-daemon[849]: reaped child process 1380, status = succeeded5198lb # [ 119.541536] niks3-server[1146]: 2026/09/23 10:36:13 INFO Received uploads request method=POST path=/api/pending_closures5199worker2 # [ 119.556445] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.202Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5200worker2 # [ 119.558359] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.202Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5201lb # [ 119.591859] niks3-server[1146]: 2026/09/23 10:36:13 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5202lb # [ 119.603673] niks3-server[1146]: 2026/09/23 10:36:13 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5203worker2 # [ 119.610477] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.258Z level=INFO msg="Uploading 1 narinfos"5204lb # [ 119.617554] niks3-server[1146]: 2026/09/23 10:36:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5205lb # [ 119.626936] niks3-server[1146]: 2026/09/23 10:36:13 INFO Signed narinfos id=38 count=15206lb # [ 119.638737] niks3-server[1146]: 2026/09/23 10:36:13 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5207lb # [ 119.645912] niks3-server[1146]: 2026/09/23 10:36:13 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5208lb # [ 119.651842] niks3-server[1146]: 2026/09/23 10:36:13 INFO Completed upload id=385209worker2 # [ 119.651730] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.299Z level=INFO msg="Upload complete. (312ms)"5210lb # [ 119.655625] niks3-server[1146]: 2026/09/23 10:36:13 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5211worker2 # [ 119.677609] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54040 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15212worker2 # [ 119.694502] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:13Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05213worker2 # [ 119.700268] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:13Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15214worker2 # [ 119.706097] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540405215worker2 # [ 119.711215] nix-daemon[849]: reaped child process 1359, status = succeeded5216worker2 # [ 119.713219] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540405217worker2 # [ 119.723369] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.371Z level=INFO msg="All 1 paths already cached"5218client # copying 1 paths to 'lb:50051'...5219client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5220worker2 # [ 119.733301] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540405221worker2 # [ 119.748895] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5222worker2 # [ 119.890199] nix-daemon[849]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)5223worker2 # [ 119.905956] nix-daemon[849]: reaped child process 1389, status = succeeded5224lb # [ 119.912278] niks3-server[1146]: 2026/09/23 10:36:13 INFO Received uploads request method=POST path=/api/pending_closures5225worker2 # [ 119.923235] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.571Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5226worker2 # [ 119.924880] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.571Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5227lb # [ 119.945501] niks3-server[1146]: 2026/09/23 10:36:13 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5228worker2 # [ 119.955099] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.602Z level=INFO msg="Uploading 1 narinfos"5229lb # [ 119.958991] niks3-server[1146]: 2026/09/23 10:36:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5230lb # [ 119.969167] niks3-server[1146]: 2026/09/23 10:36:13 INFO Signed narinfos id=39 count=15231lb # [ 119.971748] niks3-server[1146]: 2026/09/23 10:36:13 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5232lb # [ 119.984621] niks3-server[1146]: 2026/09/23 10:36:13 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5233lb # [ 119.988940] niks3-server[1146]: 2026/09/23 10:36:13 INFO Completed upload id=395234worker2 # [ 119.988385] nix-grpc-daemon[1319]: time=2026-09-23T10:36:13.636Z level=INFO msg="Upload complete. (229ms)"5235lb # [ 119.991769] niks3-server[1146]: 2026/09/23 10:36:13 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5236worker2 # [ 119.998787] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54040 duration_s=0 paths=1 nar_bytes_in=6565237worker2 # [ 120.012219] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540405238worker2 # [ 120.021009] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5239worker2 # [ 120.024076] nix-daemon[849]: reaped child process 1384, status = succeeded5240client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5241client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5242worker2 # [ 120.038289] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5243client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5244client # farm-b> LOG-farm-b-after-niks3-restart5245worker2 # [ 121.267350] nix-daemon[849]: reaped child process 1393, status = succeeded5246worker2 # [ 121.372775] nix-daemon[849]: accepted connection from pid 1409, user nix-grpc-daemon (trusted)5247worker2 # [ 121.389136] nix-daemon[849]: reaped child process 1412, status = succeeded5248lb # [ 121.394900] niks3-server[1146]: 2026/09/23 10:36:15 INFO Received uploads request method=POST path=/api/pending_closures5249worker2 # [ 121.404079] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.051Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5250worker2 # [ 121.405690] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.051Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5251worker2 # [ 121.440544] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.088Z level=INFO msg="Uploading 1 narinfos"5252lb # [ 121.433567] niks3-server[1146]: 2026/09/23 10:36:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5253lb # [ 121.444367] niks3-server[1146]: 2026/09/23 10:36:15 INFO Signed narinfos id=40 count=15254lb # [ 121.446151] niks3-server[1146]: 2026/09/23 10:36:15 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5255lb # [ 121.454450] niks3-server[1146]: 2026/09/23 10:36:15 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5256lb # [ 121.460649] niks3-server[1146]: 2026/09/23 10:36:15 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5257lb # [ 121.467014] niks3-server[1146]: 2026/09/23 10:36:15 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5258lb # [ 121.473068] niks3-server[1146]: 2026/09/23 10:36:15 INFO Completed upload id=405259worker2 # [ 121.472418] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.120Z level=INFO msg="Upload complete. (224ms)"5260lb # [ 121.475697] niks3-server[1146]: 2026/09/23 10:36:15 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5261worker2 # [ 121.483951] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54040 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15262worker2 # [ 121.501368] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:15Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05263worker2 # [ 121.505856] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:15Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15264worker2 # [ 121.511444] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540405265worker2 # [ 121.516290] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540405266worker2 # [ 121.520705] nix-daemon[849]: reaped child process 1391, status = succeeded5267worker2 # [ 121.523694] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.171Z level=INFO msg="All 2 paths already cached"5268client # copying 1 paths to 'lb:50051'...5269client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5270worker2 # [ 121.532186] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540405271worker2 # [ 121.541449] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5272worker2 # [ 121.667326] nix-daemon[849]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5273worker2 # [ 121.684453] nix-daemon[849]: reaped child process 1421, status = succeeded5274lb # [ 121.690465] niks3-server[1146]: 2026/09/23 10:36:15 INFO Received uploads request method=POST path=/api/pending_closures5275worker2 # [ 121.700081] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.348Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5276worker2 # [ 121.701553] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.348Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5277lb # [ 121.725371] niks3-server[1146]: 2026/09/23 10:36:15 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5278worker2 # [ 121.737486] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.385Z level=INFO msg="Uploading 1 narinfos"5279lb # [ 121.738964] niks3-server[1146]: 2026/09/23 10:36:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5280lb # [ 121.749257] niks3-server[1146]: 2026/09/23 10:36:15 INFO Signed narinfos id=41 count=15281lb # [ 121.751520] niks3-server[1146]: 2026/09/23 10:36:15 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5282lb # [ 121.765371] niks3-server[1146]: 2026/09/23 10:36:15 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5283lb # [ 121.770466] niks3-server[1146]: 2026/09/23 10:36:15 INFO Completed upload id=415284worker2 # [ 121.770203] nix-grpc-daemon[1319]: time=2026-09-23T10:36:15.418Z level=INFO msg="Upload complete. (217ms)"5285lb # [ 121.775385] niks3-server[1146]: 2026/09/23 10:36:15 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5286worker2 # [ 121.779479] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54040 duration_s=0 paths=1 nar_bytes_in=7845287worker2 # [ 121.791698] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540405288worker2 # [ 121.798674] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5289worker2 # [ 121.803516] nix-daemon[849]: reaped child process 1416, status = succeeded5290client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5291worker2 # [ 121.816919] nix-daemon[849]: accepted connection from pid 1259, user nix-grpc-daemon (trusted)5292client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5293client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5294client # farm-top> LOG-farm-top-after-niks3-restart5295worker2 # [ 122.693183] nix-daemon[849]: reaped child process 1425, status = succeeded5296worker2 # [ 122.802924] nix-daemon[849]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5297worker2 # [ 122.821608] nix-daemon[849]: reaped child process 1444, status = succeeded5298lb # [ 122.827693] niks3-server[1146]: 2026/09/23 10:36:16 INFO Received uploads request method=POST path=/api/pending_closures5299worker2 # [ 122.839128] nix-grpc-daemon[1319]: time=2026-09-23T10:36:16.486Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5300worker2 # [ 122.841061] nix-grpc-daemon[1319]: time=2026-09-23T10:36:16.486Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5301lb # [ 122.866893] niks3-server[1146]: 2026/09/23 10:36:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5302worker2 # [ 122.877580] nix-grpc-daemon[1319]: time=2026-09-23T10:36:16.525Z level=INFO msg="Uploading 1 narinfos"5303lb # [ 122.886174] niks3-server[1146]: 2026/09/23 10:36:16 INFO Signed narinfos id=42 count=15304lb # [ 122.893654] niks3-server[1146]: 2026/09/23 10:36:16 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5305lb # [ 122.900581] niks3-server[1146]: 2026/09/23 10:36:16 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5306lb # [ 122.903928] niks3-server[1146]: 2026/09/23 10:36:16 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5307lb # [ 122.911527] niks3-server[1146]: 2026/09/23 10:36:16 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5308lb # [ 122.917396] niks3-server[1146]: 2026/09/23 10:36:16 INFO Completed upload id=425309lb # [ 122.920222] niks3-server[1146]: 2026/09/23 10:36:16 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5310worker2 # [ 122.916929] nix-grpc-daemon[1319]: time=2026-09-23T10:36:16.565Z level=INFO msg="Upload complete. (288ms)"5311worker2 # [ 122.933389] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54040 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15312worker2 # [ 122.949154] nix-daemon[849]: reaped child process 1423, status = succeeded5313client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.93 seconds)5314client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5315client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5316(finished: subtest: a niks3 restart does not cost the leader its role, in 6.60 seconds)5317subtest: scheduler down: the other node takes the lock and keeps it5318lb: must succeed: curl -sf localhost:9901/clusters5319lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5320lb: must succeed: curl -sf localhost:9901/clusters5321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5322lb: must succeed: curl -sf localhost:9901/clusters5323lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5324lb: must succeed: curl -sf localhost:9901/clusters5325lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5326worker2 # [ 123.389223] systemd[1]: Stopping Nix worker-protocol over gRPC...5327worker2 # [ 124.161885] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:17Z level=info event=signal_exit5328lb # [ 124.678240] niks3-server[1146]: 2026/09/23 10:36:18 INFO lead: released remote=192.168.1.4:409765329worker1 # [ 125.740825] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:19Z level=info event=scheduler_restarting addr=lb:500515330worker2 # [ 125.681776] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:19Z level=info event=scheduler_restarting5331worker2 # [ 125.688228] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:19Z level=info event=scheduler_restarting addr=lb:500515332worker2 # [ 125.692992] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540505333worker2 # [ 125.698034] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:19Z level=info event=worker_gone addr=192.168.1.4:500515334worker2 # [ 125.703462] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540505335worker2 # [ 125.712441] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:19Z level=info event=worker_gone addr=192.168.1.3:500515336worker2 # [ 125.729897] nix-daemon[849]: reaped child process 1311, status = succeeded5337worker1 # [ 125.940306] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:19Z level=info event=scheduler_connected addr=lb:500515338worker1 # [ 125.945861] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5339worker2 # [ 125.897558] nix-grpc-daemon[1259]: ts=2026-09-23T10:36:19Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5340worker1 # [ 126.146704] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:19Z level=info event=scheduler_connected addr=lb:500515341worker1 # [ 126.151949] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5342worker1 # [ 126.352741] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:19Z level=info event=scheduler_connected addr=lb:500515343worker1 # [ 126.357783] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5344worker1 # [ 126.558484] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 126.563615] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker1 # [ 126.766036] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_connected addr=lb:500515347worker1 # [ 126.770164] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5348worker1 # [ 126.966325] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_connected addr=lb:500515349worker1 # [ 126.971286] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5350worker2 # [ 126.943707] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5351worker2 # [ 126.950921] systemd[1]: Stopped Nix worker-protocol over gRPC.5352worker2 # [ 126.956950] systemd[1]: nix-grpc-daemon.service: Consumed 1.378s CPU time over 16.927s wall clock time, 38.9M memory peak, 85.4K incoming IP traffic, 130K outgoing IP traffic.5353worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5354worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5355worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5356worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5357worker1 # [ 127.171824] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_connected addr=lb:500515358worker1 # [ 127.175917] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5359worker1 # [ 127.376983] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_connected addr=lb:500515360worker1 # [ 127.382007] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5361worker1 # [ 127.582670] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_connected addr=lb:500515362worker1 # [ 127.587767] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5363lb # [ 127.672931] niks3-server[1146]: 2026/09/23 10:36:21 INFO lead: acquired remote=192.168.1.3:529925364worker1 # [ 127.736237] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_take_over5365worker1 # [ 127.790098] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_connected addr=lb:500515366worker1 # [ 127.794157] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367worker1 # [ 127.990642] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_connected addr=lb:500515368worker1 # [ 127.995712] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5369worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5370worker1 # [ 128.196963] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_connected addr=lb:500515371worker1 # [ 128.203973] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5372worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5373worker1 # [ 128.404553] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:22Z level=info event=scheduler_connected addr=lb:500515374worker1 # [ 128.409582] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375lb # [ 128.357732] envoy[895]: conn peer=[2001:db8:1::1]:40256 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9749 tx=4455 ms=56855376lb # [ 128.362591] envoy[895]: conn peer=[2001:db8:1::4]:53694 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=16223 tx=1124 ms=156785377worker1 # [ 128.610293] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:22Z level=info event=scheduler_connected addr=lb:500515378worker1 # [ 128.615384] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5379worker1 # [ 128.824147] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:22Z level=info event=scheduler_connected addr=lb:500515380worker1 # [ 128.832521] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:392805381worker1 # [ 128.837811] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05382worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5383worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5384client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25385worker1 # [ 129.506005] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355725386client # don't know how to build these paths:5387client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5388worker1 # [ 129.516557] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355725389worker1 # [ 129.544124] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05390worker1 # [ 129.550025] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15391worker1 # [ 129.560234] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355725392worker1 # [ 129.570994] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355725393client # copying 1 paths to 'lb:50051'...5394client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5395worker1 # [ 129.585955] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355725396worker1 # [ 129.601352] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5397worker1 # [ 129.630694] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=info event=niks3_push_started pid=12995398worker1 # [ 129.776243] nix-grpc-daemon[1299]: time=2026-09-23T10:36:23.419Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5399worker1 # [ 129.882280] nix-daemon[851]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5400worker1 # [ 129.897703] nix-daemon[851]: reaped child process 1314, status = succeeded5401lb # [ 129.845984] niks3-server[1146]: 2026/09/23 10:36:23 INFO Received uploads request method=POST path=/api/pending_closures5402worker1 # [ 129.918466] nix-grpc-daemon[1299]: time=2026-09-23T10:36:23.562Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5403worker1 # [ 129.919925] nix-grpc-daemon[1299]: time=2026-09-23T10:36:23.562Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5404worker1 # [ 129.960886] nix-grpc-daemon[1299]: time=2026-09-23T10:36:23.605Z level=INFO msg="Uploading 1 narinfos"5405lb # [ 129.888766] niks3-server[1146]: 2026/09/23 10:36:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5406lb # [ 129.901666] niks3-server[1146]: 2026/09/23 10:36:23 INFO Signed narinfos id=43 count=15407lb # [ 129.910047] niks3-server[1146]: 2026/09/23 10:36:23 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5408lb # [ 129.919076] niks3-server[1146]: 2026/09/23 10:36:23 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5409lb # [ 129.928192] niks3-server[1146]: 2026/09/23 10:36:23 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5410lb # [ 129.934513] niks3-server[1146]: 2026/09/23 10:36:23 INFO Completed upload id=435411worker1 # [ 129.997331] nix-grpc-daemon[1299]: time=2026-09-23T10:36:23.642Z level=INFO msg="Upload complete. (222ms)"5412lb # [ 129.938904] niks3-server[1146]: 2026/09/23 10:36:23 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5413worker1 # [ 130.004844] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35572 duration_s=0 paths=1 nar_bytes_in=5205414worker1 # [ 130.020150] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355725415worker1 # [ 130.028419] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5416worker1 # [ 130.031072] nix-daemon[851]: reaped child process 1297, status = succeeded5417client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5418worker1 # [ 130.041296] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5419client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5420client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5421client # farm-a> LOG-farm-a-on-standby5422worker1 # [ 130.941383] nix-daemon[851]: reaped child process 1318, status = succeeded5423worker1 # [ 131.014579] nix-daemon[851]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5424worker1 # [ 131.033437] nix-daemon[851]: reaped child process 1337, status = succeeded5425lb # [ 130.969811] niks3-server[1146]: 2026/09/23 10:36:24 INFO Received uploads request method=POST path=/api/pending_closures5426worker1 # [ 131.043659] nix-grpc-daemon[1299]: time=2026-09-23T10:36:24.686Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5427worker1 # [ 131.046522] nix-grpc-daemon[1299]: time=2026-09-23T10:36:24.686Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5428lb # [ 131.005475] niks3-server[1146]: 2026/09/23 10:36:24 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5429lb # [ 131.019244] niks3-server[1146]: 2026/09/23 10:36:24 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5430lb # [ 131.030164] niks3-server[1146]: 2026/09/23 10:36:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5431worker1 # [ 131.097933] nix-grpc-daemon[1299]: time=2026-09-23T10:36:24.742Z level=INFO msg="Uploading 1 narinfos"5432lb # [ 131.041331] niks3-server[1146]: 2026/09/23 10:36:24 INFO Signed narinfos id=44 count=15433lb # [ 131.043924] niks3-server[1146]: 2026/09/23 10:36:24 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5434lb # [ 131.055900] niks3-server[1146]: 2026/09/23 10:36:24 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5435lb # [ 131.061673] niks3-server[1146]: 2026/09/23 10:36:24 INFO Completed upload id=445436worker1 # [ 131.123679] nix-grpc-daemon[1299]: time=2026-09-23T10:36:24.768Z level=INFO msg="Upload complete. (270ms)"5437lb # [ 131.064610] niks3-server[1146]: 2026/09/23 10:36:24 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5438worker1 # [ 131.145125] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35572 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15439worker1 # [ 131.159144] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:24Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05440worker1 # [ 131.164450] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:24Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15441worker1 # [ 131.169680] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355725442worker1 # [ 131.174404] nix-daemon[851]: reaped child process 1316, status = succeeded5443worker1 # [ 131.176167] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355725444worker1 # [ 131.183991] nix-grpc-daemon[1299]: time=2026-09-23T10:36:24.829Z level=INFO msg="All 1 paths already cached"5445client # copying 1 paths to 'lb:50051'...5446client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5447worker1 # [ 131.192684] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355725448worker1 # [ 131.202655] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5449worker1 # [ 131.337208] nix-daemon[851]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5450worker1 # [ 131.355267] nix-daemon[851]: reaped child process 1346, status = succeeded5451lb # [ 131.295864] niks3-server[1146]: 2026/09/23 10:36:24 INFO Received uploads request method=POST path=/api/pending_closures5452worker1 # [ 131.367524] nix-grpc-daemon[1299]: time=2026-09-23T10:36:25.012Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5453worker1 # [ 131.369180] nix-grpc-daemon[1299]: time=2026-09-23T10:36:25.012Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5454lb # [ 131.323204] niks3-server[1146]: 2026/09/23 10:36:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5455worker1 # [ 131.391629] nix-grpc-daemon[1299]: time=2026-09-23T10:36:25.036Z level=INFO msg="Uploading 1 narinfos"5456lb # [ 131.335993] niks3-server[1146]: 2026/09/23 10:36:24 INFO Signed narinfos id=45 count=15457lb # [ 131.345235] niks3-server[1146]: 2026/09/23 10:36:24 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5458lb # [ 131.358625] niks3-server[1146]: 2026/09/23 10:36:24 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5459worker1 # [ 131.428043] nix-grpc-daemon[1299]: time=2026-09-23T10:36:25.073Z level=INFO msg="Upload complete. (214ms)"5460lb # [ 131.366023] niks3-server[1146]: 2026/09/23 10:36:25 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5461lb # [ 131.375391] niks3-server[1146]: 2026/09/23 10:36:25 INFO Completed upload id=455462lb # [ 131.378164] niks3-server[1146]: 2026/09/23 10:36:25 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5463worker1 # [ 131.436152] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35572 duration_s=0 paths=1 nar_bytes_in=6405464worker1 # [ 131.449808] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355725465worker1 # [ 131.458052] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5466worker1 # [ 131.461574] nix-daemon[851]: reaped child process 1341, status = succeeded5467client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5468client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5469worker1 # [ 131.473640] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5470client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5471client # farm-b> LOG-farm-b-on-standby5472worker1 # [ 132.380159] nix-daemon[851]: reaped child process 1350, status = succeeded5473worker1 # [ 132.452395] nix-daemon[851]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5474worker1 # [ 132.472883] nix-daemon[851]: reaped child process 1369, status = succeeded5475lb # [ 132.418114] niks3-server[1146]: 2026/09/23 10:36:26 INFO Received uploads request method=POST path=/api/pending_closures5476worker1 # [ 132.489319] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.133Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5477worker1 # [ 132.491165] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.133Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5478lb # [ 132.456621] niks3-server[1146]: 2026/09/23 10:36:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5479worker1 # [ 132.526800] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.171Z level=INFO msg="Uploading 1 narinfos"5480lb # [ 132.475812] niks3-server[1146]: 2026/09/23 10:36:26 INFO Signed narinfos id=46 count=15481lb # [ 132.491143] niks3-server[1146]: 2026/09/23 10:36:26 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5482lb # [ 132.505361] niks3-server[1146]: 2026/09/23 10:36:26 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5483worker1 # [ 132.572913] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.218Z level=INFO msg="Upload complete. (289ms)"5484lb # [ 132.515199] niks3-server[1146]: 2026/09/23 10:36:26 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5485lb # [ 132.521781] niks3-server[1146]: 2026/09/23 10:36:26 INFO Completed upload id=465486lb # [ 132.523388] niks3-server[1146]: 2026/09/23 10:36:26 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5487lb # [ 132.526836] niks3-server[1146]: 2026/09/23 10:36:26 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5488worker1 # [ 132.590069] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35572 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15489worker1 # [ 132.612397] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:26Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05490worker1 # [ 132.622710] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:26Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15491worker1 # [ 132.629706] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355725492worker1 # [ 132.635731] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355725493worker1 # [ 132.641935] nix-daemon[851]: reaped child process 1348, status = succeeded5494client # copying 1 paths to 'lb:50051'...5495worker1 # [ 132.644215] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.285Z level=INFO msg="All 2 paths already cached"5496client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5497worker1 # [ 132.648317] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355725498worker1 # [ 132.664310] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5499worker1 # [ 132.788708] nix-daemon[851]: accepted connection from pid 1375, user nix-grpc-daemon (trusted)5500worker1 # [ 132.806881] nix-daemon[851]: reaped child process 1378, status = succeeded5501lb # [ 132.748705] niks3-server[1146]: 2026/09/23 10:36:26 INFO Received uploads request method=POST path=/api/pending_closures5502worker1 # [ 132.824254] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5503worker1 # [ 132.825984] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.469Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5504lb # [ 132.782731] niks3-server[1146]: 2026/09/23 10:36:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5505worker1 # [ 132.853047] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.497Z level=INFO msg="Uploading 1 narinfos"5506lb # [ 132.799051] niks3-server[1146]: 2026/09/23 10:36:26 INFO Signed narinfos id=47 count=15507lb # [ 132.807456] niks3-server[1146]: 2026/09/23 10:36:26 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5508lb # [ 132.822052] niks3-server[1146]: 2026/09/23 10:36:26 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5509lb # [ 132.827872] niks3-server[1146]: 2026/09/23 10:36:26 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5510lb # [ 132.831627] niks3-server[1146]: 2026/09/23 10:36:26 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5511lb # [ 132.837431] niks3-server[1146]: 2026/09/23 10:36:26 INFO Completed upload id=475512worker1 # [ 132.899038] nix-grpc-daemon[1299]: time=2026-09-23T10:36:26.544Z level=INFO msg="Upload complete. (228ms)"5513worker1 # [ 132.909642] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35572 duration_s=0 paths=1 nar_bytes_in=7685514worker1 # [ 132.924754] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355725515worker1 # [ 132.938488] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5516worker1 # [ 132.945302] nix-daemon[851]: reaped child process 1373, status = succeeded5517client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5518worker1 # [ 132.956439] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5519client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5520client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5521client # farm-top> LOG-farm-top-on-standby5522worker1 # [ 134.428901] nix-daemon[851]: reaped child process 1382, status = succeeded5523worker1 # [ 134.541450] nix-daemon[851]: accepted connection from pid 1398, user nix-grpc-daemon (trusted)5524worker1 # [ 134.562587] nix-daemon[851]: reaped child process 1401, status = succeeded5525lb # [ 134.499775] niks3-server[1146]: 2026/09/23 10:36:28 INFO Received uploads request method=POST path=/api/pending_closures5526worker1 # [ 134.572485] nix-grpc-daemon[1299]: time=2026-09-23T10:36:28.217Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5527worker1 # [ 134.575934] nix-grpc-daemon[1299]: time=2026-09-23T10:36:28.217Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5528lb # [ 134.539638] niks3-server[1146]: 2026/09/23 10:36:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5529worker1 # [ 134.610629] nix-grpc-daemon[1299]: time=2026-09-23T10:36:28.255Z level=INFO msg="Uploading 1 narinfos"5530lb # [ 134.554970] niks3-server[1146]: 2026/09/23 10:36:28 INFO Signed narinfos id=48 count=15531lb # [ 134.561259] niks3-server[1146]: 2026/09/23 10:36:28 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5532lb # [ 134.570007] niks3-server[1146]: 2026/09/23 10:36:28 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5533lb # [ 134.574767] niks3-server[1146]: 2026/09/23 10:36:28 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5534worker1 # [ 134.649805] nix-grpc-daemon[1299]: time=2026-09-23T10:36:28.294Z level=INFO msg="Upload complete. (277ms)"5535lb # [ 134.589268] niks3-server[1146]: 2026/09/23 10:36:28 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5536lb # [ 134.597699] niks3-server[1146]: 2026/09/23 10:36:28 INFO Completed upload id=485537lb # [ 134.604143] niks3-server[1146]: 2026/09/23 10:36:28 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5538worker1 # [ 134.663258] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35572 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15539worker1 # [ 134.683645] nix-daemon[851]: reaped child process 1380, status = succeeded5540client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.41 seconds)5541client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5542client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5543worker2 # [ 134.893694] systemd[1]: Starting scheduler-token.service...5544worker2 # [ 134.965439] systemd[1]: scheduler-token.service: Deactivated successfully.5545worker2 # [ 134.972568] systemd[1]: Finished scheduler-token.service.5546worker2 # [ 134.974248] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 75ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5547worker2 # [ 134.991052] systemd[1]: Starting Nix worker-protocol over gRPC...5548worker2 # [ 135.106049] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5549worker2 # [ 135.108239] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5550worker2 # [ 135.111951] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15551worker2 # [ 135.117514] nix-grpc-daemon[1465]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5552worker2 # [ 135.118969] nix-grpc-daemon[1465]: E0000 00:00:1790159788.764873 1465 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5553worker2 # [ 135.122716] nix-grpc-daemon[1465]: E0000 00:00:1790159788.770745 1465 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5554lb: must succeed: curl -sf localhost:9901/clusters5555worker2 # [ 135.133551] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:28Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5556worker2 # [ 135.164107] systemd[1]: Started Nix worker-protocol over gRPC.5557worker2 # [ 135.164975] nix-grpc-daemon[1465]: E0000 00:00:1790159788.784212 1508 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5558worker2 # [ 135.169729] nix-grpc-daemon[1465]: E0000 00:00:1790159788.799301 1508 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5559worker2 # [ 135.172493] nix-grpc-daemon[1465]: E0000 00:00:1790159788.799429 1508 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5560worker2 # [ 135.175065] nix-grpc-daemon[1465]: E0000 00:00:1790159788.817085 1508 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5561worker2 # [ 135.182915] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:28Z level=info event=scheduler_connected addr=lb:500515562worker1 # [ 135.254678] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:28Z level=info event=oidc_jwks provider=mock keys=15563worker1 # [ 135.259256] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:392805564lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5565worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5566worker1 # [ 135.264700] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05567worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5568lb: must succeed: curl -sf localhost:9901/clusters5569lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5570lb: must succeed: curl -sf localhost:9901/clusters5571lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5572worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5573worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5574(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.28 seconds)5575subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5576client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5577client # [ 135.490821] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5578client # Running as unit: rs.service; invocation ID: 589e8ce8f68d4ea89f66388d212140f55579client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5580client # [ 135.693436] nix[1398]: don't know how to build these paths:5581client # [ 135.694576] nix[1398]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5582worker1 # [ 135.756568] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355725583worker1 # [ 135.771413] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05584worker1 # [ 135.778560] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15585worker1 # [ 135.788087] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355725586worker1 # [ 135.798787] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355725587client # [ 135.746331] nix[1398]: copying 1 paths to 'lb:50051'...5588client # [ 135.749551] nix[1398]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5589worker1 # [ 135.815204] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355725590worker1 # [ 135.830306] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5591worker1 # [ 135.921610] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:549585592worker1 # [ 135.989301] nix-daemon[851]: accepted connection from pid 1414, user nix-grpc-daemon (trusted)5593worker1 # [ 136.009171] nix-daemon[851]: reaped child process 1417, status = succeeded5594lb # [ 135.954599] niks3-server[1146]: 2026/09/23 10:36:29 INFO Received uploads request method=POST path=/api/pending_closures5595worker1 # [ 136.026009] nix-grpc-daemon[1299]: time=2026-09-23T10:36:29.670Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5596worker1 # [ 136.027636] nix-grpc-daemon[1299]: time=2026-09-23T10:36:29.670Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5597lb # [ 135.984505] niks3-server[1146]: 2026/09/23 10:36:29 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5598worker1 # [ 136.058345] nix-grpc-daemon[1299]: time=2026-09-23T10:36:29.703Z level=INFO msg="Uploading 1 narinfos"5599lb # [ 136.000336] niks3-server[1146]: 2026/09/23 10:36:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5600lb # [ 136.009518] niks3-server[1146]: 2026/09/23 10:36:29 INFO Signed narinfos id=49 count=15601lb # [ 136.011755] niks3-server[1146]: 2026/09/23 10:36:29 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5602worker1 # [ 136.085946] nix-grpc-daemon[1299]: time=2026-09-23T10:36:29.731Z level=INFO msg="Upload complete. (239ms)"5603lb # [ 136.023134] niks3-server[1146]: 2026/09/23 10:36:29 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5604lb # [ 136.027310] niks3-server[1146]: 2026/09/23 10:36:29 INFO Completed upload id=495605lb # [ 136.030478] niks3-server[1146]: 2026/09/23 10:36:29 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5606worker1 # [ 136.096260] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35572 duration_s=0 paths=1 nar_bytes_in=4165607worker1 # [ 136.110944] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355725608worker1 # [ 136.117396] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5609worker1 # [ 136.122626] nix-daemon[851]: reaped child process 1412, status = succeeded5610worker1 # [ 136.133162] nix-daemon[851]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5611client # [ 136.067665] nix[1398]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5612lb: must succeed: curl -sf localhost:9901/clusters5613lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5614lb: must succeed: curl -sf localhost:9901/clusters5615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5616lb: must succeed: curl -sf localhost:9901/clusters5617lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5618lb: must succeed: curl -sf localhost:9901/clusters5619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5620worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5621worker2 # [ 138.320240] systemd-journald[262]: Received client request to rotate journal, rotating.5622worker2 # [ 138.340707] systemd-journald[262]: Vacuuming done, freed 0B of archived journals from /var/log/journal/b00527d59c8a4873901a11a975a46a10.5623worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5624worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5625lb # [ 138.356784] envoy[895]: conn peer=[2001:db8:1::1]:42042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9589 tx=4400 ms=51705626worker1 # [ 138.453108] systemd[1]: Stopping Nix worker-protocol over gRPC...5627worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5628client: must succeed: sleep 5.05629worker1 # [ 139.436658] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:33Z level=info event=draining5630client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5631worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5632worker1 # [ 143.769147] nix-daemon[851]: reaped child process 1421, status = succeeded5633worker1 # [ 143.879145] nix-daemon[851]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5634worker1 # [ 143.898320] nix-daemon[851]: reaped child process 1465, status = succeeded5635lb # [ 143.841001] niks3-server[1146]: 2026/09/23 10:36:37 INFO Received uploads request method=POST path=/api/pending_closures5636worker1 # [ 143.916376] nix-grpc-daemon[1299]: time=2026-09-23T10:36:37.559Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5637worker1 # [ 143.917852] nix-grpc-daemon[1299]: time=2026-09-23T10:36:37.559Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5638lb # [ 143.888895] niks3-server[1146]: 2026/09/23 10:36:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5639worker1 # [ 143.957982] nix-grpc-daemon[1299]: time=2026-09-23T10:36:37.602Z level=INFO msg="Uploading 1 narinfos"5640lb # [ 143.903983] niks3-server[1146]: 2026/09/23 10:36:37 INFO Signed narinfos id=50 count=15641lb # [ 143.910723] niks3-server[1146]: 2026/09/23 10:36:37 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5642lb # [ 143.921406] niks3-server[1146]: 2026/09/23 10:36:37 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5643worker1 # [ 143.988670] nix-grpc-daemon[1299]: time=2026-09-23T10:36:37.633Z level=INFO msg="Upload complete. (239ms)"5644lb # [ 143.928565] niks3-server[1146]: 2026/09/23 10:36:37 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5645lb # [ 143.934237] niks3-server[1146]: 2026/09/23 10:36:37 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5646lb # [ 143.938762] niks3-server[1146]: 2026/09/23 10:36:37 INFO Completed upload id=505647lb # [ 143.941754] niks3-server[1146]: 2026/09/23 10:36:37 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5648worker1 # [ 144.001780] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35572 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15649worker1 # [ 144.017396] nix-daemon[851]: reaped child process 1419, status = succeeded5650client # [ 143.973879] systemd[1]: rs.service: Deactivated successfully.5651client # [ 143.979024] systemd[1]: rs.service: Consumed 189ms CPU time over 8.484s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.5652worker1 # [ 144.438151] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:38Z level=info event=signal_exit5653lb # [ 144.676262] niks3-server[1146]: 2026/09/23 10:36:38 INFO lead: released remote=192.168.1.3:529925654lb # [ 145.136767] niks3-server[1146]: 2026/09/23 10:36:38 INFO lead: acquired remote=192.168.1.4:570185655worker2 # [ 145.136924] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:38Z level=info event=scheduler_take_over5656worker2 # [ 145.677837] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:39Z level=info event=scheduler_restarting addr=lb:500515657worker1 # [ 145.741671] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:39Z level=info event=scheduler_restarting5658lb # [ 145.678067] envoy[895]: [2026-09-23 10:36:39.326][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5659worker1 # [ 145.746173] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:39Z level=info event=scheduler_restarting addr=lb:500515660worker1 # [ 145.755469] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:392805661worker1 # [ 145.761329] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:39Z level=info event=worker_gone addr=192.168.1.3:500515662worker1 # [ 145.765540] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:392805663worker1 # [ 145.770852] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:39Z level=info event=worker_gone addr=192.168.1.4:500515664worker1 # [ 145.796937] nix-daemon[851]: reaped child process 1272, status = succeeded5665worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.17 seconds)5666client: waiting for success: ! systemctl is-active rs5667worker2 # [ 145.879215] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:39Z level=info event=scheduler_connected addr=lb:500515668worker2 # [ 145.906181] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:39Z level=info event=oidc_jwks provider=mock keys=15669worker1 # [ 145.969275] nix-grpc-daemon[1220]: ts=2026-09-23T10:36:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5670worker2 # [ 145.910635] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:356165671worker2 # [ 145.915992] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05672client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5673client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5674client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5675client: must fail: journalctl -u rs | grep -q 'reconnecting'5676client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5677lb: must succeed: curl -sf localhost:9901/clusters5678worker2 # [ 146.056125] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:39Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:356305679lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5680worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5681worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5682lb: must succeed: curl -sf localhost:9901/clusters5683lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5684worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5685worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5686worker1 # [ 147.369986] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5687worker1 # [ 147.376357] systemd[1]: Stopped Nix worker-protocol over gRPC.5688worker1 # [ 147.379116] systemd[1]: nix-grpc-daemon.service: Consumed 1.571s CPU time over 32.117s wall clock time, 39.3M memory peak, 275.9K incoming IP traffic, 154.1K outgoing IP traffic.5689worker1 # [ 147.400365] systemd[1]: Starting Nix worker-protocol over gRPC...5690worker1 # [ 147.554003] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5691worker1 # [ 147.556247] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5692worker1 # [ 147.559939] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15693worker1 # [ 147.566580] nix-grpc-daemon[1473]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5694worker1 # [ 147.568378] nix-grpc-daemon[1473]: E0000 00:00:1790159801.209728 1473 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5695worker1 # [ 147.582109] nix-grpc-daemon[1473]: E0000 00:00:1790159801.211655 1473 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5696worker1 # [ 147.608245] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:41Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5697worker1 # [ 147.609931] nix-grpc-daemon[1473]: E0000 00:00:1790159801.222327 1514 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5698worker1 # [ 147.615043] nix-grpc-daemon[1473]: E0000 00:00:1790159801.240015 1514 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5699worker1 # [ 147.617733] nix-grpc-daemon[1473]: E0000 00:00:1790159801.240166 1514 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5700worker1 # [ 147.624372] systemd[1]: Started Nix worker-protocol over gRPC.5701worker1 # [ 147.626997] nix-grpc-daemon[1473]: E0000 00:00:1790159801.256540 1514 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5702worker1 # [ 147.629546] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:41Z level=info event=scheduler_connected addr=lb:500515703worker2 # [ 147.566114] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:41Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:356165704worker2 # [ 147.572229] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:41Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05705lb: must succeed: curl -sf localhost:9901/clusters5706lb # [ 148.359596] envoy[895]: conn peer=[2001:db8:1::1]:47788 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1516 ms=82655707lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5708worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5709worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5710(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.00 seconds)5711subtest: scheduler restart mid-build: build finishes, no double build5712client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5713client # [ 148.493046] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5714client # Running as unit: mid.service; invocation ID: 1e9bcbbf88f9455e97d15214ea965bd55715client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5716worker1 # [ 148.755594] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608085717worker1 # [ 148.770453] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5718client # [ 148.724503] nix[1434]: don't know how to build these paths:5719client # [ 148.725946] nix[1434]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5720worker2 # [ 148.738050] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:42Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05721worker2 # [ 148.743780] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:42Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15722worker2 # [ 148.763897] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356365723worker2 # [ 148.777674] nix-daemon[849]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5724worker2 # [ 148.801449] nix-grpc-daemon[1465]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 1/5)5725worker2 # [ 148.834176] nix-grpc-daemon[1465]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 2/5)5726worker2 # [ 148.916975] nix-grpc-daemon[1465]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 265 ms (attempt 3/5)5727lb # [ 148.364513] envoy[895]: conn peer=[2001:db8:1::3]:40522 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3662 tx=1191 ms=304805728worker2 # [ 149.183964] nix-grpc-daemon[1465]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 674 ms (attempt 4/5)5729worker2 # [ 149.869592] nix-grpc-daemon[1465]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5730worker2 # [ 149.869838] nix-grpc-daemon[1465]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5731worker2 # [ 149.893817] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356365732client # [ 149.909039] nix[1434]: copying 1 paths to 'lb:50051'...5733worker2 # [ 149.909611] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:356365734client # [ 149.909350] nix[1434]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5735worker2 # [ 149.924717] nix-daemon[849]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5736worker2 # [ 149.956152] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:43Z level=info event=niks3_push_started pid=15945737worker2 # [ 150.100204] nix-grpc-daemon[1594]: time=2026-09-23T10:36:43.748Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5738worker2 # [ 150.206227] nix-daemon[849]: accepted connection from pid 1607, user nix-grpc-daemon (trusted)5739worker2 # [ 150.224852] nix-daemon[849]: reaped child process 1610, status = succeeded5740lb # [ 150.224600] niks3-server[1146]: 2026/09/23 10:36:43 INFO Received uploads request method=POST path=/api/pending_closures5741worker2 # [ 150.235775] nix-grpc-daemon[1594]: time=2026-09-23T10:36:43.882Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5742worker2 # [ 150.239762] nix-grpc-daemon[1594]: time=2026-09-23T10:36:43.883Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5743lb # [ 150.274774] niks3-server[1146]: 2026/09/23 10:36:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5744lb # [ 150.292341] niks3-server[1146]: 2026/09/23 10:36:43 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5745lb # [ 150.304400] niks3-server[1146]: 2026/09/23 10:36:43 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5746lb # [ 150.311615] niks3-server[1146]: 2026/09/23 10:36:43 INFO Signed narinfos id=51 count=15747worker2 # [ 150.313141] nix-grpc-daemon[1594]: time=2026-09-23T10:36:43.959Z level=INFO msg="Uploading 1 narinfos"5748lb # [ 150.321244] niks3-server[1146]: 2026/09/23 10:36:43 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5749lb # [ 150.332652] niks3-server[1146]: 2026/09/23 10:36:43 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5750lb # [ 150.341583] niks3-server[1146]: 2026/09/23 10:36:43 INFO Completed upload id=515751worker2 # [ 150.343933] nix-grpc-daemon[1594]: time=2026-09-23T10:36:43.992Z level=INFO msg="Upload complete. (240ms)"5752worker2 # [ 150.356148] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35636 duration_s=0 paths=1 nar_bytes_in=4245753worker2 # [ 150.371918] nix-grpc-daemon[1465]: ts=2026-09-23T10:36:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356365754worker2 # [ 150.378899] nix-daemon[849]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5755worker2 # [ 150.384320] nix-daemon[849]: reaped child process 1592, status = succeeded5756client # [ 150.392204] nix[1434]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5757worker2 # [ 150.396336] nix-daemon[849]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5758lb: must succeed: curl -sf localhost:9901/clusters5759lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5760lb: must succeed: curl -sf localhost:9901/clusters5761lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5762worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5763worker2 # [ 152.348811] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1465 (nix-grpc-daemon) on client request.5764worker2 # [ 152.350397] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5765lb # [ 152.365778] niks3-server[1146]: 2026/09/23 10:36:46 INFO lead: released remote=192.168.1.4:570185766lb # [ 152.371539] envoy[895]: [2026-09-23 10:36:46.020][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5767worker2 # [ 152.379567] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5768worker2 # [ 152.387147] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5769client # [ 152.382025] nix[1434]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5770worker1 # [ 152.446368] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5771client # [ 152.388443] nix[1434]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5772worker2 # [ 152.399808] systemd[1]: nix-grpc-daemon.service: Consumed 514ms CPU time over 17.391s wall clock time, 35M memory peak, 218.9K incoming IP traffic, 59.9K outgoing IP traffic.5773worker2 # [ 152.402243] systemd[1]: Starting scheduler-token.service...5774worker2 # [ 152.427543] nix-daemon[849]: reaped child process 1582, status = succeeded5775worker2 # [ 152.429602] nix-daemon[849]: reaped child process 1612, status = succeeded5776worker2 # [ 152.436480] nix-daemon[1614]: unexpected Nix daemon error: error: interrupted by the user5777worker2 # [ 152.443753] nix-daemon[849]: reaped child process 1614, status = failed with exit code 15778worker2 # [ 152.464259] systemd[1]: scheduler-token.service: Deactivated successfully.5779worker2 # [ 152.466744] systemd[1]: Finished scheduler-token.service.5780worker2 # [ 152.467518] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 77ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5781worker2 # [ 152.472964] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5782worker2 # [ 152.476155] systemd[1]: Starting scheduler-token.service...5783lb # [ 152.514731] niks3-server[1146]: 2026/09/23 10:36:46 INFO lead: acquired remote=192.168.1.3:536285784worker1 # [ 152.577452] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_take_over5785worker2 # [ 152.523147] systemd[1]: scheduler-token.service: Deactivated successfully.5786worker2 # [ 152.525378] systemd[1]: Finished scheduler-token.service.5787worker2 # [ 152.526157] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 52ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5788worker2 # [ 152.530935] systemd[1]: Starting Nix worker-protocol over gRPC...5789worker1 # [ 152.646876] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_connected addr=lb:500515790worker2 # [ 152.628604] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5791worker2 # [ 152.630424] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5792worker1 # [ 152.688751] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. retried and the latest reset reason: remote connection failure"5793worker2 # [ 152.634115] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15794worker2 # [ 152.640064] nix-grpc-daemon[1657]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5795worker2 # [ 152.641507] nix-grpc-daemon[1657]: E0000 00:00:1790159806.286782 1657 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5796worker2 # [ 152.644555] nix-grpc-daemon[1657]: E0000 00:00:1790159806.292671 1657 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5797worker2 # [ 152.647806] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:46Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5798worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.34 seconds)5799client: must succeed: sleep 5.05800worker2 # [ 152.664081] systemd[1]: Started Nix worker-protocol over gRPC.5801worker2 # [ 152.672131] nix-grpc-daemon[1657]: E0000 00:00:1790159806.305713 1700 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5802worker2 # [ 152.691389] nix-grpc-daemon[1657]: E0000 00:00:1790159806.324307 1700 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5803worker2 # [ 152.694379] nix-grpc-daemon[1657]: E0000 00:00:1790159806.324520 1700 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5804worker2 # [ 152.696982] nix-grpc-daemon[1657]: E0000 00:00:1790159806.338905 1700 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5805worker2 # [ 152.704631] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:46Z level=info event=scheduler_connected addr=lb:500515806worker1 # [ 152.778904] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=oidc_jwks provider=mock keys=15807worker1 # [ 152.783401] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:581145808worker1 # [ 152.788988] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05809worker1 # [ 152.889587] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_connected addr=lb:500515810worker1 # [ 152.896265] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5811worker1 # [ 152.919472] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:581305812worker1 # [ 152.956229] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05813worker1 # [ 152.962248] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15814worker2 # [ 152.917624] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588865815worker2 # [ 152.928480] nix-daemon[849]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)5816worker2 # [ 152.950771] nix-daemon[849]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)5817client # [ 152.966242] nix[1434]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5818worker2 # [ 152.970160] nix-daemon[849]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)5819worker1 # [ 153.097225] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_connected addr=lb:500515820worker1 # [ 153.102503] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5821worker1 # [ 153.303239] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=scheduler_connected addr=lb:500515822worker1 # [ 153.309629] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:581145823worker1 # [ 153.320292] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05824client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5825client: waiting for success: ! systemctl is-active mid5826worker2 # [ 157.989748] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:51Z level=info event=niks3_push_started pid=17385827worker2 # [ 158.011205] nix-daemon[849]: reaped child process 1713, status = succeeded5828worker2 # [ 158.113845] nix-grpc-daemon[1738]: time=2026-09-23T10:36:51.761Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5829worker2 # [ 158.220398] nix-daemon[849]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)5830worker2 # [ 158.236864] nix-daemon[849]: reaped child process 1753, status = succeeded5831lb # [ 158.245932] niks3-server[1146]: 2026/09/23 10:36:51 INFO Received uploads request method=POST path=/api/pending_closures5832worker2 # [ 158.256107] nix-grpc-daemon[1738]: time=2026-09-23T10:36:51.903Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5833worker2 # [ 158.257568] nix-grpc-daemon[1738]: time=2026-09-23T10:36:51.903Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5834lb # [ 158.304235] niks3-server[1146]: 2026/09/23 10:36:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5835worker2 # [ 158.311287] nix-grpc-daemon[1738]: time=2026-09-23T10:36:51.958Z level=INFO msg="Uploading 1 narinfos"5836lb # [ 158.319318] niks3-server[1146]: 2026/09/23 10:36:51 INFO Signed narinfos id=52 count=15837lb # [ 158.329288] niks3-server[1146]: 2026/09/23 10:36:51 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5838lb # [ 158.347454] niks3-server[1146]: 2026/09/23 10:36:51 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5839lb # [ 158.354544] niks3-server[1146]: 2026/09/23 10:36:52 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5840worker2 # [ 158.360296] nix-grpc-daemon[1738]: time=2026-09-23T10:36:52.008Z level=INFO msg="Upload complete. (242ms)"5841lb # [ 158.363438] niks3-server[1146]: 2026/09/23 10:36:52 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5842lb # [ 158.366909] niks3-server[1146]: 2026/09/23 10:36:52 INFO Completed upload id=525843lb # [ 158.371027] niks3-server[1146]: 2026/09/23 10:36:52 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5844lb # [ 158.377365] envoy[895]: conn peer=[2001:db8:1::4]:57200 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1948 tx=328 ms=172085845worker2 # [ 158.372295] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58886 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15846worker2 # [ 158.389125] nix-daemon[849]: reaped child process 1711, status = succeeded5847client # [ 158.409437] systemd[1]: mid.service: Deactivated successfully.5848client # [ 158.414550] systemd[1]: mid.service: Consumed 194ms CPU time over 9.918s wall clock time, 41.5M memory peak, 8.5K incoming IP traffic, 10.2K outgoing IP traffic.5849client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5850client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5851client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5852(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.70 seconds)5853subtest: interrupting the client stops the build on the worker5854client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5855client # [ 159.167810] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5856client # Running as unit: intr.service; invocation ID: ce3bf5928a6a4dabad54e8dd83915a915857client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5858worker2 # [ 159.342218] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588865859client # [ 159.347500] nix[1470]: don't know how to build these paths:5860client # [ 159.348493] nix[1470]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5861worker1 # [ 159.429477] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:53Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05862worker1 # [ 159.440767] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:53Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15863worker1 # [ 159.451382] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581505864worker1 # [ 159.467422] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5)5865worker1 # [ 159.562090] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 2/5)5866worker1 # [ 159.576466] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 212 ms (attempt 3/5)5867worker1 # [ 159.797939] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 557 ms (attempt 4/5)5868worker1 # [ 160.357468] nix-grpc-daemon[1473]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5869worker1 # [ 160.363589] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5870worker1 # [ 160.380753] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581505871client # [ 160.331455] nix[1470]: copying 1 paths to 'lb:50051'...5872worker1 # [ 160.396088] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581505873client # [ 160.331811] nix[1470]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5874worker1 # [ 160.418081] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5875worker1 # [ 160.460719] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:54Z level=info event=niks3_push_started pid=15735876worker1 # [ 160.614971] nix-grpc-daemon[1573]: time=2026-09-23T10:36:54.259Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5877worker1 # [ 160.719014] nix-daemon[851]: accepted connection from pid 1586, user nix-grpc-daemon (trusted)5878worker1 # [ 160.735248] nix-daemon[851]: reaped child process 1589, status = succeeded5879lb # [ 160.682598] niks3-server[1146]: 2026/09/23 10:36:54 INFO Received uploads request method=POST path=/api/pending_closures5880worker1 # [ 160.758068] nix-grpc-daemon[1573]: time=2026-09-23T10:36:54.402Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5881worker1 # [ 160.759570] nix-grpc-daemon[1573]: time=2026-09-23T10:36:54.402Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5882worker1 # [ 160.796837] nix-grpc-daemon[1573]: time=2026-09-23T10:36:54.440Z level=INFO msg="Uploading 1 narinfos"5883lb # [ 160.725217] niks3-server[1146]: 2026/09/23 10:36:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign5884lb # [ 160.738303] niks3-server[1146]: 2026/09/23 10:36:54 INFO Signed narinfos id=53 count=15885lb # [ 160.747238] niks3-server[1146]: 2026/09/23 10:36:54 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5886lb # [ 160.756673] niks3-server[1146]: 2026/09/23 10:36:54 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5887worker1 # [ 160.835230] nix-grpc-daemon[1573]: time=2026-09-23T10:36:54.480Z level=INFO msg="Upload complete. (217ms)"5888lb # [ 160.772981] niks3-server[1146]: 2026/09/23 10:36:54 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete5889lb # [ 160.780202] niks3-server[1146]: 2026/09/23 10:36:54 INFO Completed upload id=535890worker1 # [ 160.842844] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=4245891lb # [ 160.783722] niks3-server[1146]: 2026/09/23 10:36:54 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5892worker1 # [ 160.855624] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581505893worker1 # [ 160.860507] nix-daemon[851]: reaped child process 1570, status = succeeded5894worker1 # [ 160.867168] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5895worker1 # [ 160.880176] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5896client # [ 160.814544] nix[1470]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5897client: must succeed: systemctl kill -s INT intr5898worker1 # [ 162.953336] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:56Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5899client # [ 162.883965] systemd[1]: intr.service: Sent signal SIGINT to main process 1470 (nix) on client request.5900client # [ 162.893359] nix[1470]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5901client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5902client # [ 162.904441] nix[1470]: error: interrupted by the user5903client # [ 162.929107] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5904client # [ 162.934444] systemd[1]: intr.service: Failed with result 'exit-code'.5905client # [ 162.937006] systemd[1]: intr.service: Consumed 168ms CPU time over 3.764s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.5906worker1 # [ 163.106833] nix-daemon[1593]: unexpected Nix daemon error: error: interrupted by the user5907worker1 # [ 163.115510] nix-daemon[851]: reaped child process 1591, status = succeeded5908worker1 # [ 163.117014] nix-daemon[851]: reaped child process 1593, status = failed with exit code 15909(finished: subtest: interrupting the client stops the build on the worker, in 5.04 seconds)5910subtest: a deploy mid-build drains: the build finishes, the new generation takes over5911client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5912client # [ 164.240951] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5913client # Running as unit: sw.service; invocation ID: fdb9d170bede408eaeeb33c3ea8d3f2b5914client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)5915client # [ 164.436127] nix[1491]: don't know how to build these paths:5916client # [ 164.437220] nix[1491]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5917worker1 # [ 164.500320] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581505918worker1 # [ 164.515121] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:58Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05919worker1 # [ 164.521249] nix-grpc-daemon[1473]: ts=2026-09-23T10:36:58Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15920worker2 # [ 164.462837] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588865921worker2 # [ 164.480461] nix-grpc-daemon[1657]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)5922worker2 # [ 164.523517] nix-grpc-daemon[1657]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 2/5)5923worker2 # [ 164.598442] nix-grpc-daemon[1657]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 180 ms (attempt 3/5)5924worker2 # [ 164.780609] nix-grpc-daemon[1657]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 4/5)5925worker2 # [ 165.119886] nix-grpc-daemon[1657]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5926worker2 # [ 165.126974] nix-grpc-daemon[1657]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5927worker2 # [ 165.144600] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588865928client # [ 165.156144] nix[1491]: copying 1 paths to 'lb:50051'...5929client # [ 165.161770] nix[1491]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5930worker2 # [ 165.163326] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588865931worker2 # [ 165.178595] nix-daemon[849]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)5932worker2 # [ 165.340890] nix-daemon[849]: accepted connection from pid 1789, user nix-grpc-daemon (trusted)5933worker2 # [ 165.358041] nix-daemon[849]: reaped child process 1792, status = succeeded5934lb # [ 165.364711] niks3-server[1146]: 2026/09/23 10:36:59 INFO Received uploads request method=POST path=/api/pending_closures5935worker2 # [ 165.379939] nix-grpc-daemon[1738]: time=2026-09-23T10:36:59.026Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5936worker2 # [ 165.382307] nix-grpc-daemon[1738]: time=2026-09-23T10:36:59.026Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5937lb # [ 165.400798] niks3-server[1146]: 2026/09/23 10:36:59 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5938worker2 # [ 165.411084] nix-grpc-daemon[1738]: time=2026-09-23T10:36:59.058Z level=INFO msg="Uploading 1 narinfos"5939lb # [ 165.412653] niks3-server[1146]: 2026/09/23 10:36:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign5940lb # [ 165.423920] niks3-server[1146]: 2026/09/23 10:36:59 INFO Signed narinfos id=54 count=15941lb # [ 165.432919] niks3-server[1146]: 2026/09/23 10:36:59 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5942worker2 # [ 165.441852] nix-grpc-daemon[1738]: time=2026-09-23T10:36:59.090Z level=INFO msg="Upload complete. (244ms)"5943lb # [ 165.445271] niks3-server[1146]: 2026/09/23 10:36:59 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete5944lb # [ 165.451326] niks3-server[1146]: 2026/09/23 10:36:59 INFO Completed upload id=545945worker2 # [ 165.452313] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58886 duration_s=0 paths=1 nar_bytes_in=4165946lb # [ 165.455319] niks3-server[1146]: 2026/09/23 10:36:59 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5947worker2 # [ 165.471534] nix-grpc-daemon[1657]: ts=2026-09-23T10:36:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588865948worker2 # [ 165.481290] nix-daemon[849]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)5949worker2 # [ 165.485512] nix-daemon[849]: reaped child process 1787, status = succeeded5950client # [ 165.502026] nix[1491]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5951worker2 # [ 165.507488] nix-daemon[849]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)5952worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25953worker2 # Checking switch inhibitors... done5954worker2 # [ 167.244763] nixos[1828]: switching to system configuration /nix/store/4npjvmffyc6cdz7lfv1dl2m7zszml4z9-nixos-system-worker2-test5955worker2 # activating the configuration...5956worker2 # [ 167.250692] systemd[1]: Stopped target Remote File Systems.5957worker2 # [ 167.251516] systemd[1]: Stopped target Local File Systems.5958worker2 # setting up /etc...5959worker2 # [ 167.788781] systemd[1]: Reload requested from client PID 1828 ('.switch-to-conf') (unit backdoor.service)...5960worker2 # [ 167.790785] systemd[1]: Reloading...5961lb # [ 168.364596] envoy[895]: conn peer=[2001:db8:1::1]:45130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3897 tx=1925 ms=97045962lb # [ 168.369463] envoy[895]: conn peer=[2001:db8:1::1]:37164 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=850 ms=35735963worker2 # [ 172.634472] systemd[1]: Reloading finished in 4838 ms.5964worker2 # restarting sysinit-reactivation.target5965worker2 # [ 172.763012] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5966worker2 # [ 172.890336] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5967worker2 # [ 172.892510] systemd[1]: Reached target Reactivate sysinit units.5968worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5969worker2 # [ 172.903407] dbus-broker-launch[486]: Caught SIGHUP, trigger reload.5970worker2 # [ 172.904948] systemd[1]: Reloading D-Bus System Message Bus...5971worker2 # [ 172.905783] systemd[1]: Reloading Nix worker-protocol over gRPC...5972worker2 # [ 172.917360] dbus-broker-launch[486]: Looking up NSS user entry for 'systemd-timesync'...5973worker2 # [ 172.918462] dbus-broker-launch[486]: NSS returned no entry for 'systemd-timesync'5974worker2 # [ 172.919429] dbus-broker-launch[486]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5975worker2 # [ 172.944751] systemd[1]: Reloaded Nix worker-protocol over gRPC.5976worker2 # [ 172.962016] systemd[1]: Reloaded D-Bus System Message Bus.5977worker2 # [ 172.966193] systemd[1]: Reached target Remote File Systems.5978worker2 # [ 172.968573] systemd[1]: Starting linger-users.service...5979worker2 # [ 172.980067] systemd[1]: Reached target Local File Systems.5980worker2 # [ 172.980859] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5981worker2 # [ 172.982084] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5982worker2 # [ 172.995328] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5983worker2 # [ 173.000812] systemd[1]: Starting Create SUID/SGID Wrappers...5984worker2 # [ 173.001667] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5985worker2 # [ 173.008148] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675986worker2 # [ 173.010295] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5987worker2 # [ 173.022464] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5988worker2 # [ 173.029445] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5989worker2 # [ 173.043007] systemd[1]: linger-users.service: Deactivated successfully.5990worker2 # [ 173.043955] systemd[1]: Finished linger-users.service.5991worker2 # [ 173.335300] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5992worker2 # [ 173.340293] systemd[1]: Finished Create SUID/SGID Wrappers.5993worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5994worker2 # [ 173.591125] nixos[1828]: finished switching to system configuration /nix/store/4npjvmffyc6cdz7lfv1dl2m7zszml4z9-nixos-system-worker2-test5995worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.79 seconds)5996worker2: must succeed: pgrep -f 'read -t [9]0 x'5997worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)5998client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&25999worker2 # [ 173.661160] nix-grpc-daemon[1657]: ts=2026-09-23T10:37:07Z level=info event=draining6000client # don't know how to build these paths:6001client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6002worker1 # [ 173.940936] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581506003worker1 # [ 173.949792] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506004worker1 # [ 173.970735] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06005worker1 # [ 173.979868] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16006worker1 # [ 173.988473] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506007worker1 # [ 173.995752] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506008client # copying 1 paths to 'lb:50051'...6009worker1 # [ 174.007530] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506010client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6011worker1 # [ 174.028752] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6012worker1 # [ 174.180243] nix-daemon[851]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6013worker1 # [ 174.201729] nix-daemon[851]: reaped child process 1651, status = succeeded6014lb # [ 174.138827] niks3-server[1146]: 2026/09/23 10:37:07 INFO Received uploads request method=POST path=/api/pending_closures6015worker1 # [ 174.212133] nix-grpc-daemon[1573]: time=2026-09-23T10:37:07.856Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6016worker1 # [ 174.213606] nix-grpc-daemon[1573]: time=2026-09-23T10:37:07.856Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6017lb # [ 174.171390] niks3-server[1146]: 2026/09/23 10:37:07 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6018worker1 # [ 174.246520] nix-grpc-daemon[1573]: time=2026-09-23T10:37:07.891Z level=INFO msg="Uploading 1 narinfos"6019lb # [ 174.186859] niks3-server[1146]: 2026/09/23 10:37:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6020lb # [ 174.196911] niks3-server[1146]: 2026/09/23 10:37:07 INFO Signed narinfos id=55 count=16021lb # [ 174.202537] niks3-server[1146]: 2026/09/23 10:37:07 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6022lb # [ 174.211614] niks3-server[1146]: 2026/09/23 10:37:07 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6023lb # [ 174.216105] niks3-server[1146]: 2026/09/23 10:37:07 INFO Completed upload id=556024worker1 # [ 174.278181] nix-grpc-daemon[1573]: time=2026-09-23T10:37:07.923Z level=INFO msg="Upload complete. (233ms)"6025lb # [ 174.218783] niks3-server[1146]: 2026/09/23 10:37:07 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6026worker1 # [ 174.288206] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=5286027worker1 # [ 174.301806] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506028worker1 # [ 174.308558] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6029worker1 # [ 174.313787] nix-daemon[851]: reaped child process 1646, status = succeeded6030client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6031worker1 # [ 174.326037] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6032client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6033client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6034client # farm-a> LOG-farm-a-during-drain6035worker1 # [ 175.163148] nix-daemon[851]: reaped child process 1655, status = succeeded6036worker1 # [ 175.271925] nix-daemon[851]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)6037worker1 # [ 175.291855] nix-daemon[851]: reaped child process 1674, status = succeeded6038lb # [ 175.235729] niks3-server[1146]: 2026/09/23 10:37:08 INFO Received uploads request method=POST path=/api/pending_closures6039worker1 # [ 175.308094] nix-grpc-daemon[1573]: time=2026-09-23T10:37:08.951Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6040worker1 # [ 175.309598] nix-grpc-daemon[1573]: time=2026-09-23T10:37:08.951Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6041lb # [ 175.290878] niks3-server[1146]: 2026/09/23 10:37:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6042worker1 # [ 175.359436] nix-grpc-daemon[1573]: time=2026-09-23T10:37:09.004Z level=INFO msg="Uploading 1 narinfos"6043lb # [ 175.301221] niks3-server[1146]: 2026/09/23 10:37:08 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6044lb # [ 175.313646] niks3-server[1146]: 2026/09/23 10:37:08 INFO Signed narinfos id=56 count=16045worker1 # [ 175.381468] nix-grpc-daemon[1573]: time=2026-09-23T10:37:09.026Z level=INFO msg="Upload complete. (246ms)"6046lb # [ 175.320337] niks3-server[1146]: 2026/09/23 10:37:08 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6047lb # [ 175.323804] niks3-server[1146]: 2026/09/23 10:37:08 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6048lb # [ 175.331963] niks3-server[1146]: 2026/09/23 10:37:08 INFO Completed upload id=566049worker1 # [ 175.393726] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16050lb # [ 175.334349] niks3-server[1146]: 2026/09/23 10:37:08 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6051lb # [ 175.338706] niks3-server[1146]: 2026/09/23 10:37:08 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6052worker1 # [ 175.411508] nix-daemon[851]: reaped child process 1653, status = succeeded6053worker1 # [ 175.415759] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:09Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06054worker1 # [ 175.422493] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:09Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16055worker1 # [ 175.428937] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506056worker1 # [ 175.434208] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506057worker1 # [ 175.442017] nix-grpc-daemon[1573]: time=2026-09-23T10:37:09.087Z level=INFO msg="All 1 paths already cached"6058client # copying 1 paths to 'lb:50051'...6059worker1 # [ 175.450696] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506060client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6061worker1 # [ 175.471807] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6062worker1 # [ 175.610585] nix-daemon[851]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6063worker1 # [ 175.625904] nix-daemon[851]: reaped child process 1683, status = succeeded6064lb # [ 175.569424] niks3-server[1146]: 2026/09/23 10:37:09 INFO Received uploads request method=POST path=/api/pending_closures6065worker1 # [ 175.640808] nix-grpc-daemon[1573]: time=2026-09-23T10:37:09.285Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6066worker1 # [ 175.642308] nix-grpc-daemon[1573]: time=2026-09-23T10:37:09.285Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6067lb # [ 175.596270] niks3-server[1146]: 2026/09/23 10:37:09 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6068lb # [ 175.609888] niks3-server[1146]: 2026/09/23 10:37:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6069worker1 # [ 175.678709] nix-grpc-daemon[1573]: time=2026-09-23T10:37:09.323Z level=INFO msg="Uploading 1 narinfos"6070lb # [ 175.621815] niks3-server[1146]: 2026/09/23 10:37:09 INFO Signed narinfos id=57 count=16071lb # [ 175.626573] niks3-server[1146]: 2026/09/23 10:37:09 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6072lb # [ 175.635669] niks3-server[1146]: 2026/09/23 10:37:09 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6073worker1 # [ 175.702237] nix-grpc-daemon[1573]: time=2026-09-23T10:37:09.347Z level=INFO msg="Upload complete. (217ms)"6074lb # [ 175.641035] niks3-server[1146]: 2026/09/23 10:37:09 INFO Completed upload id=576075lb # [ 175.644873] niks3-server[1146]: 2026/09/23 10:37:09 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6076worker1 # [ 175.711835] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=6406077worker1 # [ 175.722218] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506078worker1 # [ 175.731463] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6079worker1 # [ 175.734265] nix-daemon[851]: reaped child process 1678, status = succeeded6080client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6081worker1 # [ 175.745198] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6082client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6083client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6084client # farm-b> LOG-farm-b-during-drain6085worker1 # [ 177.208513] nix-daemon[851]: reaped child process 1687, status = succeeded6086worker1 # [ 177.337235] nix-daemon[851]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)6087worker1 # [ 177.358443] nix-daemon[851]: reaped child process 1706, status = succeeded6088lb # [ 177.295794] niks3-server[1146]: 2026/09/23 10:37:10 INFO Received uploads request method=POST path=/api/pending_closures6089worker1 # [ 177.368469] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.012Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6090worker1 # [ 177.369932] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.012Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6091lb # [ 177.336699] niks3-server[1146]: 2026/09/23 10:37:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6092worker1 # [ 177.409666] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.054Z level=INFO msg="Uploading 1 narinfos"6093lb # [ 177.349408] niks3-server[1146]: 2026/09/23 10:37:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6094lb # [ 177.362265] niks3-server[1146]: 2026/09/23 10:37:10 INFO Signed narinfos id=58 count=16095lb # [ 177.368969] niks3-server[1146]: 2026/09/23 10:37:11 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6096lb # [ 177.379970] niks3-server[1146]: 2026/09/23 10:37:11 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6097worker1 # [ 177.442519] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.087Z level=INFO msg="Upload complete. (284ms)"6098lb # [ 177.385681] niks3-server[1146]: 2026/09/23 10:37:11 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6099lb # [ 177.389575] niks3-server[1146]: 2026/09/23 10:37:11 INFO Completed upload id=586100lb # [ 177.392145] niks3-server[1146]: 2026/09/23 10:37:11 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6101worker1 # [ 177.458949] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16102worker1 # [ 177.478666] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:11Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06103worker1 # [ 177.487598] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:11Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16104worker1 # [ 177.495082] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506105worker1 # [ 177.501907] nix-daemon[851]: reaped child process 1685, status = succeeded6106worker1 # [ 177.505879] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506107client # copying 1 paths to 'lb:50051'...6108worker1 # [ 177.511796] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.155Z level=INFO msg="All 2 paths already cached"6109client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6110worker1 # [ 177.516978] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506111worker1 # [ 177.534425] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6112worker1 # [ 177.671668] nix-daemon[851]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6113worker1 # [ 177.686759] nix-daemon[851]: reaped child process 1715, status = succeeded6114lb # [ 177.631441] niks3-server[1146]: 2026/09/23 10:37:11 INFO Received uploads request method=POST path=/api/pending_closures6115worker1 # [ 177.702727] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.347Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6116worker1 # [ 177.704275] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.347Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6117lb # [ 177.661857] niks3-server[1146]: 2026/09/23 10:37:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6118worker1 # [ 177.731883] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.376Z level=INFO msg="Uploading 1 narinfos"6119lb # [ 177.675589] niks3-server[1146]: 2026/09/23 10:37:11 INFO Signed narinfos id=59 count=16120lb # [ 177.683297] niks3-server[1146]: 2026/09/23 10:37:11 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6121lb # [ 177.690640] niks3-server[1146]: 2026/09/23 10:37:11 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6122lb # [ 177.699737] niks3-server[1146]: 2026/09/23 10:37:11 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6123worker1 # [ 177.766545] nix-grpc-daemon[1573]: time=2026-09-23T10:37:11.411Z level=INFO msg="Upload complete. (219ms)"6124lb # [ 177.705723] niks3-server[1146]: 2026/09/23 10:37:11 INFO Completed upload id=596125lb # [ 177.710182] niks3-server[1146]: 2026/09/23 10:37:11 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6126worker1 # [ 177.776235] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=7686127worker1 # [ 177.790019] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506128worker1 # [ 177.798710] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6129worker1 # [ 177.801900] nix-daemon[851]: reaped child process 1710, status = succeeded6130client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6131worker1 # [ 177.815107] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6132client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6133client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6134client # farm-top> LOG-farm-top-during-drain6135worker1 # [ 179.076218] nix-daemon[851]: reaped child process 1719, status = succeeded6136worker1 # [ 179.198412] nix-daemon[851]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6137worker1 # [ 179.213353] nix-daemon[851]: reaped child process 1738, status = succeeded6138lb # [ 179.158865] niks3-server[1146]: 2026/09/23 10:37:12 INFO Received uploads request method=POST path=/api/pending_closures6139worker1 # [ 179.232423] nix-grpc-daemon[1573]: time=2026-09-23T10:37:12.876Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6140worker1 # [ 179.234189] nix-grpc-daemon[1573]: time=2026-09-23T10:37:12.876Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6141lb # [ 179.199245] niks3-server[1146]: 2026/09/23 10:37:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6142worker1 # [ 179.271346] nix-grpc-daemon[1573]: time=2026-09-23T10:37:12.916Z level=INFO msg="Uploading 1 narinfos"6143lb # [ 179.213867] niks3-server[1146]: 2026/09/23 10:37:12 INFO Signed narinfos id=60 count=16144lb # [ 179.221471] niks3-server[1146]: 2026/09/23 10:37:12 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6145lb # [ 179.234706] niks3-server[1146]: 2026/09/23 10:37:12 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6146worker1 # [ 179.304995] nix-grpc-daemon[1573]: time=2026-09-23T10:37:12.950Z level=INFO msg="Upload complete. (279ms)"6147lb # [ 179.244348] niks3-server[1146]: 2026/09/23 10:37:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6148lb # [ 179.248054] niks3-server[1146]: 2026/09/23 10:37:12 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6149lb # [ 179.253971] niks3-server[1146]: 2026/09/23 10:37:12 INFO Completed upload id=606150lb # [ 179.256733] niks3-server[1146]: 2026/09/23 10:37:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6151worker1 # [ 179.317462] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16152worker1 # [ 179.334833] nix-daemon[851]: reaped child process 1717, status = succeeded6153client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.66 seconds)6154client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6155client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6156worker2: must succeed: pkill -f 'read -t [9]0 x'6157worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6158client: waiting for success: ! systemctl is-active sw6159worker2 # [ 179.841478] nix-daemon[849]: reaped child process 1799, status = succeeded6160worker2 # [ 179.914783] nix-daemon[849]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)6161worker2 # [ 179.935875] nix-daemon[849]: reaped child process 1971, status = succeeded6162lb # [ 179.934445] niks3-server[1146]: 2026/09/23 10:37:13 INFO Received uploads request method=POST path=/api/pending_closures6163worker2 # [ 179.944535] nix-grpc-daemon[1738]: time=2026-09-23T10:37:13.591Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6164worker2 # [ 179.946115] nix-grpc-daemon[1738]: time=2026-09-23T10:37:13.591Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6165lb # [ 179.977169] niks3-server[1146]: 2026/09/23 10:37:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6166worker2 # [ 179.990374] nix-grpc-daemon[1738]: time=2026-09-23T10:37:13.638Z level=INFO msg="Uploading 1 narinfos"6167lb # [ 179.992497] niks3-server[1146]: 2026/09/23 10:37:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6168lb # [ 180.002831] niks3-server[1146]: 2026/09/23 10:37:13 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6169lb # [ 180.009723] niks3-server[1146]: 2026/09/23 10:37:13 INFO Signed narinfos id=61 count=16170lb # [ 180.017659] niks3-server[1146]: 2026/09/23 10:37:13 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6171worker2 # [ 180.017571] nix-grpc-daemon[1738]: time=2026-09-23T10:37:13.665Z level=INFO msg="Upload complete. (275ms)"6172lb # [ 180.023037] niks3-server[1146]: 2026/09/23 10:37:13 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6173lb # [ 180.027880] niks3-server[1146]: 2026/09/23 10:37:13 INFO Completed upload id=616174lb # [ 180.030906] niks3-server[1146]: 2026/09/23 10:37:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6175worker2 # [ 180.033256] nix-grpc-daemon[1657]: ts=2026-09-23T10:37:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58886 duration_s=14 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16176worker2 # [ 180.048698] nix-daemon[849]: reaped child process 1794, status = succeeded6177client # [ 180.068320] systemd[1]: sw.service: Deactivated successfully.6178client # [ 180.072714] systemd[1]: sw.service: Consumed 199ms CPU time over 15.832s wall clock time, 41.2M memory peak, 8K incoming IP traffic, 8.8K outgoing IP traffic.6179client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6180client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6181worker2 # [ 180.665041] nix-grpc-daemon[1657]: ts=2026-09-23T10:37:14Z level=info event=signal_exit6182client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6183worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6184worker2 # [ 180.777839] systemd[1]: Starting scheduler-token.service...6185worker2 # [ 180.840640] systemd[1]: scheduler-token.service: Deactivated successfully.6186worker2 # [ 180.845300] systemd[1]: Finished scheduler-token.service.6187worker2 # [ 180.846683] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 66ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6188worker2 # [ 182.668989] nix-grpc-daemon[1657]: ts=2026-09-23T10:37:16Z level=info event=scheduler_restarting6189worker1 # [ 182.738638] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:581146190worker1 # [ 182.746667] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:16Z level=info event=worker_gone addr=192.168.1.4:500516191worker2 # [ 182.698692] nix-grpc-daemon[1657]: ts=2026-09-23T10:37:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6192worker2 # [ 182.724532] nix-daemon[849]: reaped child process 1709, status = succeeded6193worker2 # [ 184.698285] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6194worker2 # [ 184.702974] systemd[1]: nix-grpc-daemon.service: Consumed 790ms CPU time over 32.328s wall clock time, 36.1M memory peak, 51.6K incoming IP traffic, 59.5K outgoing IP traffic.6195worker2 # [ 184.721030] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6196worker2 # [ 184.732633] systemd[1]: Starting scheduler-token.service...6197worker2 # [ 184.810144] systemd[1]: scheduler-token.service: Deactivated successfully.6198worker2 # [ 184.816525] systemd[1]: Finished scheduler-token.service.6199worker2 # [ 184.818189] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6200worker2 # [ 184.826396] systemd[1]: Starting Nix worker-protocol over gRPC...6201worker2 # [ 184.955117] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6202worker2 # [ 184.960124] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6203worker2 # [ 184.965816] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16204worker2 # [ 184.984208] nix-grpc-daemon[1992]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6205worker2 # [ 184.985622] nix-grpc-daemon[1992]: E0000 00:00:1790159838.611677 1992 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6206worker2 # [ 184.987901] nix-grpc-daemon[1992]: E0000 00:00:1790159838.613892 1992 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6207worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 4.32 seconds)6208worker2 # [ 185.015734] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:18Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6209lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6210lb: must succeed: curl -sf localhost:9901/clusters6211worker2 # [ 185.017697] nix-grpc-daemon[1992]: E0000 00:00:1790159838.623281 2033 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6212worker2 # [ 185.019985] systemd[1]: Started Nix worker-protocol over gRPC.6213worker2 # [ 185.020826] nix-grpc-daemon[1992]: E0000 00:00:1790159838.645804 2033 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6214worker2 # [ 185.026141] nix-grpc-daemon[1992]: E0000 00:00:1790159838.646421 2033 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6215worker2 # [ 185.030444] nix-grpc-daemon[1992]: E0000 00:00:1790159838.662116 2033 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6216worker2 # [ 185.033027] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:18Z level=info event=scheduler_connected addr=lb:500516217worker1 # [ 185.096932] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:18Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:581146218worker1 # [ 185.106054] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:18Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06219lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6220lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.08 seconds)6221lb: must succeed: curl -sf localhost:9901/clusters6222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6223worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6224worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6225(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 21.03 seconds)6226subtest: queue gauges show a burst while every build blocks6227lb: must succeed: curl -sf localhost:9901/clusters6228lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6229lb: must succeed: curl -sf localhost:9901/clusters6230lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6231worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6232worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6233client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6234client # [ 185.429945] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6235client # Running as unit: burst.service; invocation ID: 857e5fb9643f429da3ea4fdb667055e26236client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6237lb: must succeed: curl -sf localhost:9901/clusters6238lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6239lb: must succeed: curl -sf localhost:9901/clusters6240lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6241worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6242client # [ 185.621691] nix[1547]: don't know how to build these paths:6243client # [ 185.622649] nix[1547]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6244client # [ 185.623685] nix[1547]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6245worker1 # [ 185.684874] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581506246client # [ 185.624822] nix[1547]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6247client # [ 185.626692] nix[1547]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6248worker1 # [ 185.704462] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06249worker2 # [ 185.646793] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401666250client # [ 185.656217] nix[1547]: copying 1 paths to 'lb:50051'...6251client # [ 185.657290] nix[1547]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6252worker1 # [ 185.720674] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16253worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)6254worker2 # [ 185.661325] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6255worker1 # [ 185.729256] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06256worker1 # [ 185.738853] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16257worker1 # [ 185.750237] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06258worker2 # [ 185.685672] nix-grpc-daemon[1992]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 1/5)6259worker1 # [ 185.756249] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06260worker1 # [ 185.759674] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506261worker1 # [ 185.764288] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506262worker1 # [ 185.770205] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506263worker1 # [ 185.780708] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6264worker2 # [ 185.753722] nix-grpc-daemon[1992]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 2/5)6265worker1 # [ 185.894020] nix-daemon[851]: accepted connection from pid 1753, user nix-grpc-daemon (trusted)6266worker1 # [ 185.911758] nix-daemon[851]: reaped child process 1756, status = succeeded6267lb # [ 185.853944] niks3-server[1146]: 2026/09/23 10:37:19 INFO Received uploads request method=POST path=/api/pending_closures6268worker1 # [ 185.925657] nix-grpc-daemon[1573]: time=2026-09-23T10:37:19.570Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6269worker1 # [ 185.927741] nix-grpc-daemon[1573]: time=2026-09-23T10:37:19.572Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6270lb # [ 185.884110] niks3-server[1146]: 2026/09/23 10:37:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6271worker1 # [ 185.955190] nix-grpc-daemon[1573]: time=2026-09-23T10:37:19.600Z level=INFO msg="Uploading 1 narinfos"6272lb # [ 185.897343] niks3-server[1146]: 2026/09/23 10:37:19 INFO Signed narinfos id=62 count=16273worker2 # [ 185.900979] nix-grpc-daemon[1992]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 3/5)6274lb # [ 185.905544] niks3-server[1146]: 2026/09/23 10:37:19 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6275worker1 # [ 185.983655] nix-grpc-daemon[1573]: time=2026-09-23T10:37:19.628Z level=INFO msg="Upload complete. (220ms)"6276lb # [ 185.921734] niks3-server[1146]: 2026/09/23 10:37:19 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6277lb # [ 185.930650] niks3-server[1146]: 2026/09/23 10:37:19 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6278worker1 # [ 185.992929] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=4166279lb # [ 185.934985] niks3-server[1146]: 2026/09/23 10:37:19 INFO Completed upload id=626280lb # [ 185.937680] niks3-server[1146]: 2026/09/23 10:37:19 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6281worker1 # [ 186.005573] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506282worker1 # [ 186.011193] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6283worker1 # [ 186.015503] nix-daemon[851]: reaped child process 1751, status = succeeded6284client # [ 185.959947] nix[1547]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6285worker1 # [ 186.027301] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6286worker2 # [ 186.108950] nix-grpc-daemon[1992]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 656 ms (attempt 4/5)6287lb: must succeed: curl -sf localhost:9901/clusters6288lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6289lb: must succeed: curl -sf localhost:9901/clusters6290worker2 # [ 186.767081] nix-grpc-daemon[1992]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6291worker2 # [ 186.767400] nix-grpc-daemon[1992]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6293worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6294worker2 # [ 186.791798] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401666295client # [ 186.805193] nix[1547]: copying 1 paths to 'lb:50051'...6296worker2 # [ 186.806451] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401666297client # [ 186.805447] nix[1547]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6298worker2 # [ 186.824507] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6299worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6300worker2 # [ 186.855138] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:20Z level=info event=niks3_push_started pid=20526301worker1 # [ 187.058976] nix-daemon[851]: reaped child process 1760, status = succeeded6302worker2 # [ 187.027438] nix-grpc-daemon[2052]: time=2026-09-23T10:37:20.673Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6303worker1 # [ 187.178424] nix-daemon[851]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)6304worker1 # [ 187.196996] nix-daemon[851]: reaped child process 1787, status = succeeded6305worker2 # [ 187.135965] nix-daemon[849]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6306lb # [ 187.140857] niks3-server[1146]: 2026/09/23 10:37:20 INFO Received uploads request method=POST path=/api/pending_closures6307worker1 # [ 187.213692] nix-grpc-daemon[1573]: time=2026-09-23T10:37:20.857Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6308worker2 # [ 187.151903] nix-daemon[849]: reaped child process 2072, status = succeeded6309worker1 # [ 187.215162] nix-grpc-daemon[1573]: time=2026-09-23T10:37:20.857Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6310lb # [ 187.159307] niks3-server[1146]: 2026/09/23 10:37:20 INFO Received uploads request method=POST path=/api/pending_closures6311worker2 # [ 187.172059] nix-grpc-daemon[2052]: time=2026-09-23T10:37:20.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6312worker2 # [ 187.173518] nix-grpc-daemon[2052]: time=2026-09-23T10:37:20.819Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6313lb # [ 187.197528] niks3-server[1146]: 2026/09/23 10:37:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6314worker1 # [ 187.264958] nix-grpc-daemon[1573]: time=2026-09-23T10:37:20.909Z level=INFO msg="Uploading 1 narinfos"6315lb # [ 187.208832] niks3-server[1146]: 2026/09/23 10:37:20 INFO Signed narinfos id=63 count=16316lb # [ 187.215695] niks3-server[1146]: 2026/09/23 10:37:20 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6317lb # [ 187.222699] niks3-server[1146]: 2026/09/23 10:37:20 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6318lb # [ 187.232131] niks3-server[1146]: 2026/09/23 10:37:20 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6319lb # [ 187.235058] niks3-server[1146]: 2026/09/23 10:37:20 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6320worker1 # [ 187.301495] nix-grpc-daemon[1573]: time=2026-09-23T10:37:20.946Z level=INFO msg="Upload complete. (265ms)"6321lb # [ 187.243820] niks3-server[1146]: 2026/09/23 10:37:20 INFO Completed upload id=636322lb # [ 187.250618] niks3-server[1146]: 2026/09/23 10:37:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6323worker2 # [ 187.250014] nix-grpc-daemon[2052]: time=2026-09-23T10:37:20.896Z level=INFO msg="Uploading 1 narinfos"6324lb # [ 187.257387] niks3-server[1146]: 2026/09/23 10:37:20 INFO Signed narinfos id=64 count=16325worker1 # [ 187.314086] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=16326lb # [ 187.261100] niks3-server[1146]: 2026/09/23 10:37:20 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6327lb # [ 187.263599] niks3-server[1146]: 2026/09/23 10:37:20 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6328lb # [ 187.266805] niks3-server[1146]: 2026/09/23 10:37:20 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6329worker1 # [ 187.332723] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:20Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=16330lb # [ 187.274982] niks3-server[1146]: 2026/09/23 10:37:20 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6331worker2 # [ 187.273578] nix-grpc-daemon[2052]: time=2026-09-23T10:37:20.921Z level=INFO msg="Upload complete. (247ms)"6332lb # [ 187.278913] niks3-server[1146]: 2026/09/23 10:37:20 INFO Completed upload id=646333lb # [ 187.281153] niks3-server[1146]: 2026/09/23 10:37:20 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6334worker1 # [ 187.345535] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506335worker2 # [ 187.286713] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=0 paths=1 nar_bytes_in=4166336worker1 # [ 187.353295] nix-daemon[851]: reaped child process 1758, status = succeeded6337worker1 # [ 187.357639] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506338client # [ 187.296168] nix[1547]: copying 1 paths to 'lb:50051'...6339client # [ 187.298510] nix[1547]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6340worker1 # [ 187.363673] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506341worker2 # [ 187.304552] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401666342worker1 # [ 187.378022] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6343worker2 # [ 187.315414] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6344worker2 # [ 187.324377] nix-daemon[849]: reaped child process 2050, status = succeeded6345client # [ 187.330338] nix[1547]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6346worker2 # [ 187.334259] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6347worker1 # [ 187.510409] nix-daemon[851]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6348worker1 # [ 187.527184] nix-daemon[851]: reaped child process 1794, status = succeeded6349lb # [ 187.471751] niks3-server[1146]: 2026/09/23 10:37:21 INFO Received uploads request method=POST path=/api/pending_closures6350worker1 # [ 187.542546] nix-grpc-daemon[1573]: time=2026-09-23T10:37:21.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6351worker1 # [ 187.543999] nix-grpc-daemon[1573]: time=2026-09-23T10:37:21.187Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6352lb # [ 187.499705] niks3-server[1146]: 2026/09/23 10:37:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6353worker1 # [ 187.571701] nix-grpc-daemon[1573]: time=2026-09-23T10:37:21.216Z level=INFO msg="Uploading 1 narinfos"6354lb # [ 187.515049] niks3-server[1146]: 2026/09/23 10:37:21 INFO Signed narinfos id=65 count=16355lb # [ 187.522387] niks3-server[1146]: 2026/09/23 10:37:21 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6356worker1 # [ 187.598644] nix-grpc-daemon[1573]: time=2026-09-23T10:37:21.243Z level=INFO msg="Upload complete. (210ms)"6357lb # [ 187.538061] niks3-server[1146]: 2026/09/23 10:37:21 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6358worker1 # [ 187.608228] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=4166359lb # [ 187.549082] niks3-server[1146]: 2026/09/23 10:37:21 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6360lb # [ 187.554297] niks3-server[1146]: 2026/09/23 10:37:21 INFO Completed upload id=656361worker1 # [ 187.619207] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506362lb # [ 187.557479] niks3-server[1146]: 2026/09/23 10:37:21 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6363worker1 # [ 187.628373] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6364worker1 # [ 187.631194] nix-daemon[851]: reaped child process 1789, status = succeeded6365worker1 # [ 187.642305] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6366client # [ 187.576811] nix[1547]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6367lb # [ 188.362342] envoy[895]: conn peer=[2001:db8:1::1]:35948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9617 tx=4403 ms=54076368worker2 # [ 188.442125] nix-daemon[849]: reaped child process 2076, status = succeeded6369worker2 # [ 188.510091] nix-daemon[849]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)6370worker2 # [ 188.528384] nix-daemon[849]: reaped child process 2100, status = succeeded6371lb # [ 188.367446] envoy[895]: conn peer=[2001:db8:1::1]:36848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3029 tx=1595 ms=156146372lb # [ 188.367592] envoy[895]: conn peer=[2001:db8:1::4]:59580 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1511 tx=343 ms=299806373lb # [ 188.535084] niks3-server[1146]: 2026/09/23 10:37:22 INFO Received uploads request method=POST path=/api/pending_closures6374worker2 # [ 188.556181] nix-grpc-daemon[2052]: time=2026-09-23T10:37:22.201Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6375worker2 # [ 188.558187] nix-grpc-daemon[2052]: time=2026-09-23T10:37:22.201Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6376lb # [ 188.577288] niks3-server[1146]: 2026/09/23 10:37:22 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6377lb # [ 188.596677] niks3-server[1146]: 2026/09/23 10:37:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6378worker2 # [ 188.601869] nix-grpc-daemon[2052]: time=2026-09-23T10:37:22.249Z level=INFO msg="Uploading 1 narinfos"6379lb # [ 188.605788] niks3-server[1146]: 2026/09/23 10:37:22 INFO Signed narinfos id=66 count=16380lb # [ 188.611912] niks3-server[1146]: 2026/09/23 10:37:22 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6381lb # [ 188.618547] niks3-server[1146]: 2026/09/23 10:37:22 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6382lb # [ 188.624047] niks3-server[1146]: 2026/09/23 10:37:22 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6383lb # [ 188.631310] niks3-server[1146]: 2026/09/23 10:37:22 INFO Completed upload id=666384worker2 # [ 188.630378] nix-grpc-daemon[2052]: time=2026-09-23T10:37:22.278Z level=INFO msg="Upload complete. (281ms)"6385lb # [ 188.633918] niks3-server[1146]: 2026/09/23 10:37:22 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6386worker1 # [ 188.701387] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:22Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=16387worker2 # [ 188.645087] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=16388worker2 # [ 188.661642] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401666389worker2 # [ 188.671270] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401666390worker2 # [ 188.676673] nix-daemon[849]: reaped child process 2074, status = succeeded6391client # [ 188.679754] nix[1547]: copying 1 paths to 'lb:50051'...6392worker2 # [ 188.680303] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401666393client # [ 188.680078] nix[1547]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6394worker2 # [ 188.692766] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6395worker2 # [ 188.827532] nix-daemon[849]: accepted connection from pid 2104, user nix-grpc-daemon (trusted)6396worker2 # [ 188.847302] nix-daemon[849]: reaped child process 2107, status = succeeded6397lb # [ 188.846173] niks3-server[1146]: 2026/09/23 10:37:22 INFO Received uploads request method=POST path=/api/pending_closures6398worker2 # [ 188.855980] nix-grpc-daemon[2052]: time=2026-09-23T10:37:22.503Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6399worker2 # [ 188.857469] nix-grpc-daemon[2052]: time=2026-09-23T10:37:22.503Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6400lb # [ 188.874921] niks3-server[1146]: 2026/09/23 10:37:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6401worker2 # [ 188.881706] nix-grpc-daemon[2052]: time=2026-09-23T10:37:22.529Z level=INFO msg="Uploading 1 narinfos"6402lb # [ 188.889817] niks3-server[1146]: 2026/09/23 10:37:22 INFO Signed narinfos id=67 count=16403lb # [ 188.897088] niks3-server[1146]: 2026/09/23 10:37:22 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6404worker2 # [ 188.907023] nix-grpc-daemon[2052]: time=2026-09-23T10:37:22.555Z level=INFO msg="Upload complete. (205ms)"6405lb # [ 188.907698] niks3-server[1146]: 2026/09/23 10:37:22 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6406worker2 # [ 188.916256] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=0 paths=1 nar_bytes_in=4166407lb # [ 188.914400] niks3-server[1146]: 2026/09/23 10:37:22 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6408lb # [ 188.921086] niks3-server[1146]: 2026/09/23 10:37:22 INFO Completed upload id=676409lb # [ 188.923085] niks3-server[1146]: 2026/09/23 10:37:22 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6410worker2 # [ 188.927345] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401666411worker2 # [ 188.932301] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6412worker2 # [ 188.936375] nix-daemon[849]: reaped child process 2102, status = succeeded6413worker2 # [ 188.945628] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6414client # [ 188.943155] nix[1547]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6415worker1 # [ 189.476991] nix-daemon[851]: reaped child process 1798, status = succeeded6416worker1 # [ 189.574402] nix-daemon[851]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)6417worker1 # [ 189.590813] nix-daemon[851]: reaped child process 1826, status = succeeded6418lb # [ 189.536914] niks3-server[1146]: 2026/09/23 10:37:23 INFO Received uploads request method=POST path=/api/pending_closures6419worker1 # [ 189.612376] nix-grpc-daemon[1573]: time=2026-09-23T10:37:23.254Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6420worker1 # [ 189.613820] nix-grpc-daemon[1573]: time=2026-09-23T10:37:23.254Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6421lb # [ 189.582907] niks3-server[1146]: 2026/09/23 10:37:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6422worker1 # [ 189.653214] nix-grpc-daemon[1573]: time=2026-09-23T10:37:23.298Z level=INFO msg="Uploading 1 narinfos"6423lb # [ 189.595542] niks3-server[1146]: 2026/09/23 10:37:23 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6424lb # [ 189.604254] niks3-server[1146]: 2026/09/23 10:37:23 INFO Signed narinfos id=68 count=16425lb # [ 189.606562] niks3-server[1146]: 2026/09/23 10:37:23 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6426worker1 # [ 189.679693] nix-grpc-daemon[1573]: time=2026-09-23T10:37:23.324Z level=INFO msg="Upload complete. (223ms)"6427lb # [ 189.618056] niks3-server[1146]: 2026/09/23 10:37:23 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6428lb # [ 189.621808] niks3-server[1146]: 2026/09/23 10:37:23 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6429lb # [ 189.625614] niks3-server[1146]: 2026/09/23 10:37:23 INFO Completed upload id=686430lb # [ 189.627515] niks3-server[1146]: 2026/09/23 10:37:23 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6431worker1 # [ 189.688699] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=16432worker1 # [ 189.703189] nix-daemon[851]: reaped child process 1796, status = succeeded6433worker2 # [ 190.824742] nix-daemon[849]: reaped child process 2111, status = succeeded6434worker2 # [ 190.923285] nix-daemon[849]: accepted connection from pid 2136, user nix-grpc-daemon (trusted)6435worker2 # [ 190.942684] nix-daemon[849]: reaped child process 2139, status = succeeded6436lb # [ 190.947801] niks3-server[1146]: 2026/09/23 10:37:24 INFO Received uploads request method=POST path=/api/pending_closures6437worker2 # [ 190.957260] nix-grpc-daemon[2052]: time=2026-09-23T10:37:24.604Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6438worker2 # [ 190.958879] nix-grpc-daemon[2052]: time=2026-09-23T10:37:24.604Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6439lb # [ 190.985933] niks3-server[1146]: 2026/09/23 10:37:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6440worker2 # [ 190.997289] nix-grpc-daemon[2052]: time=2026-09-23T10:37:24.645Z level=INFO msg="Uploading 1 narinfos"6441lb # [ 191.000763] niks3-server[1146]: 2026/09/23 10:37:24 INFO Signed narinfos id=69 count=16442lb # [ 191.011313] niks3-server[1146]: 2026/09/23 10:37:24 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6443lb # [ 191.022270] niks3-server[1146]: 2026/09/23 10:37:24 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6444lb # [ 191.034108] niks3-server[1146]: 2026/09/23 10:37:24 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6445worker2 # [ 191.039662] nix-grpc-daemon[2052]: time=2026-09-23T10:37:24.687Z level=INFO msg="Upload complete. (282ms)"6446lb # [ 191.040497] niks3-server[1146]: 2026/09/23 10:37:24 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6447lb # [ 191.044234] niks3-server[1146]: 2026/09/23 10:37:24 INFO Completed upload id=696448lb # [ 191.048181] niks3-server[1146]: 2026/09/23 10:37:24 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6449worker2 # [ 191.053082] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=16450worker2 # [ 191.071873] nix-daemon[849]: reaped child process 2109, status = succeeded6451client # [ 191.086770] systemd[1]: burst.service: Deactivated successfully.6452client # [ 191.091917] systemd[1]: burst.service: Consumed 213ms CPU time over 5.658s wall clock time, 55.6M memory peak, 13.8K incoming IP traffic, 18.5K outgoing IP traffic.6453client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6454client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6455lb: must succeed: curl -sf localhost:9901/clusters6456lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6457lb: must succeed: curl -sf localhost:9901/clusters6458lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6459worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6460worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6461(finished: subtest: queue gauges show a burst while every build blocks, in 6.90 seconds)6462subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6463client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16464worker2 # [ 192.329096] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401666465worker1 # [ 192.436538] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06466worker1 # [ 192.436833] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16467worker1 # [ 192.436956] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06468worker1 # [ 192.437067] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06469worker1 # [ 192.459486] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506470worker1 # [ 192.474715] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506471worker1 # [ 192.489080] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506472worker1 # [ 192.504123] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6473worker1 # [ 192.660922] nix-daemon[851]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6474worker1 # [ 192.679068] nix-daemon[851]: reaped child process 1846, status = succeeded6475lb # [ 192.620998] niks3-server[1146]: 2026/09/23 10:37:26 INFO Received uploads request method=POST path=/api/pending_closures6476worker1 # [ 192.692966] nix-grpc-daemon[1573]: time=2026-09-23T10:37:26.337Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6477worker1 # [ 192.694451] nix-grpc-daemon[1573]: time=2026-09-23T10:37:26.337Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6478lb # [ 192.652370] niks3-server[1146]: 2026/09/23 10:37:26 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6479worker1 # [ 192.725173] nix-grpc-daemon[1573]: time=2026-09-23T10:37:26.370Z level=INFO msg="Uploading 1 narinfos"6480lb # [ 192.667761] niks3-server[1146]: 2026/09/23 10:37:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6481lb # [ 192.677501] niks3-server[1146]: 2026/09/23 10:37:26 INFO Signed narinfos id=70 count=16482lb # [ 192.682720] niks3-server[1146]: 2026/09/23 10:37:26 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6483lb # [ 192.691689] niks3-server[1146]: 2026/09/23 10:37:26 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6484lb # [ 192.697562] niks3-server[1146]: 2026/09/23 10:37:26 INFO Completed upload id=706485worker1 # [ 192.759277] nix-grpc-daemon[1573]: time=2026-09-23T10:37:26.404Z level=INFO msg="Upload complete. (239ms)"6486lb # [ 192.699586] niks3-server[1146]: 2026/09/23 10:37:26 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6487worker1 # [ 192.772330] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=5126488worker1 # [ 192.785490] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506489worker1 # [ 192.792230] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6490worker1 # [ 192.796639] nix-daemon[851]: reaped child process 1841, status = succeeded6491worker1 # [ 192.811206] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6492worker2 # [ 192.336910] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401666493worker1 # [ 194.511187] nix-daemon[851]: reaped child process 1850, status = succeeded6494worker1 # [ 194.579215] nix-daemon[851]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)6495worker1 # [ 194.597996] nix-daemon[851]: reaped child process 1869, status = succeeded6496lb # [ 194.543803] niks3-server[1146]: 2026/09/23 10:37:28 INFO Received uploads request method=POST path=/api/pending_closures6497worker1 # [ 194.616649] nix-grpc-daemon[1573]: time=2026-09-23T10:37:28.261Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6498worker1 # [ 194.618759] nix-grpc-daemon[1573]: time=2026-09-23T10:37:28.261Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6499lb # [ 194.586496] niks3-server[1146]: 2026/09/23 10:37:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6500worker1 # [ 194.658857] nix-grpc-daemon[1573]: time=2026-09-23T10:37:28.303Z level=INFO msg="Uploading 1 narinfos"6501lb # [ 194.603287] niks3-server[1146]: 2026/09/23 10:37:28 INFO Signed narinfos id=71 count=16502lb # [ 194.614961] niks3-server[1146]: 2026/09/23 10:37:28 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6503lb # [ 194.628293] niks3-server[1146]: 2026/09/23 10:37:28 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6504lb # [ 194.645164] niks3-server[1146]: 2026/09/23 10:37:28 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6505worker1 # [ 194.712006] nix-grpc-daemon[1573]: time=2026-09-23T10:37:28.357Z level=INFO msg="Upload complete. (290ms)"6506lb # [ 194.653943] niks3-server[1146]: 2026/09/23 10:37:28 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6507lb # [ 194.658255] niks3-server[1146]: 2026/09/23 10:37:28 INFO Completed upload id=716508lb # [ 194.661194] niks3-server[1146]: 2026/09/23 10:37:28 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6509worker1 # [ 194.730634] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=16510worker1 # [ 194.750060] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:28Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16511worker1 # [ 194.758494] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506512worker1 # [ 194.765808] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:28Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06513worker1 # [ 194.772816] nix-daemon[851]: reaped child process 1848, status = succeeded6514worker1 # [ 194.777038] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506515worker1 # [ 194.791322] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506516worker1 # [ 194.801435] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6517worker1 # [ 194.931719] nix-daemon[851]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6518worker1 # [ 194.946577] nix-daemon[851]: reaped child process 1878, status = succeeded6519lb # [ 194.891179] niks3-server[1146]: 2026/09/23 10:37:28 INFO Received uploads request method=POST path=/api/pending_closures6520worker1 # [ 194.962076] nix-grpc-daemon[1573]: time=2026-09-23T10:37:28.606Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6521worker1 # [ 194.963550] nix-grpc-daemon[1573]: time=2026-09-23T10:37:28.607Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6522lb # [ 194.919600] niks3-server[1146]: 2026/09/23 10:37:28 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6523worker1 # [ 194.990989] nix-grpc-daemon[1573]: time=2026-09-23T10:37:28.636Z level=INFO msg="Uploading 1 narinfos"6524lb # [ 194.932806] niks3-server[1146]: 2026/09/23 10:37:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6525lb # [ 194.944343] niks3-server[1146]: 2026/09/23 10:37:28 INFO Signed narinfos id=72 count=16526lb # [ 194.947049] niks3-server[1146]: 2026/09/23 10:37:28 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6527lb # [ 194.960242] niks3-server[1146]: 2026/09/23 10:37:28 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6528worker1 # [ 195.022986] nix-grpc-daemon[1573]: time=2026-09-23T10:37:28.668Z level=INFO msg="Upload complete. (216ms)"6529lb # [ 194.964756] niks3-server[1146]: 2026/09/23 10:37:28 INFO Completed upload id=726530lb # [ 194.968153] niks3-server[1146]: 2026/09/23 10:37:28 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6531worker1 # [ 195.032260] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=5126532worker1 # [ 195.045917] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506533worker1 # [ 195.052614] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6534worker1 # [ 195.058014] nix-daemon[851]: reaped child process 1873, status = succeeded6535worker1 # [ 195.068908] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6536worker1 # [ 196.556802] nix-daemon[851]: reaped child process 1882, status = succeeded6537worker1 # [ 196.653661] nix-daemon[851]: accepted connection from pid 1898, user nix-grpc-daemon (trusted)6538lb # [ 196.614463] niks3-server[1146]: 2026/09/23 10:37:30 INFO Received uploads request method=POST path=/api/pending_closures6539worker1 # [ 196.678866] nix-daemon[851]: reaped child process 1901, status = succeeded6540worker1 # [ 196.684536] nix-grpc-daemon[1573]: time=2026-09-23T10:37:30.328Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6541worker1 # [ 196.686047] nix-grpc-daemon[1573]: time=2026-09-23T10:37:30.328Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6542lb # [ 196.660364] niks3-server[1146]: 2026/09/23 10:37:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6543worker1 # [ 196.730484] nix-grpc-daemon[1573]: time=2026-09-23T10:37:30.375Z level=INFO msg="Uploading 1 narinfos"6544lb # [ 196.672220] niks3-server[1146]: 2026/09/23 10:37:30 INFO Signed narinfos id=73 count=16545lb # [ 196.679958] niks3-server[1146]: 2026/09/23 10:37:30 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6546lb # [ 196.688287] niks3-server[1146]: 2026/09/23 10:37:30 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6547lb # [ 196.691185] niks3-server[1146]: 2026/09/23 10:37:30 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6548lb # [ 196.704919] niks3-server[1146]: 2026/09/23 10:37:30 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6549worker1 # [ 196.766859] nix-grpc-daemon[1573]: time=2026-09-23T10:37:30.411Z level=INFO msg="Upload complete. (284ms)"6550lb # [ 196.708876] niks3-server[1146]: 2026/09/23 10:37:30 INFO Completed upload id=736551lb # [ 196.710683] niks3-server[1146]: 2026/09/23 10:37:30 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6552worker1 # [ 196.786206] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=16553worker1 # [ 196.800597] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:30Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16554worker1 # [ 196.810906] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506555worker1 # [ 196.818253] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:30Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06556worker1 # [ 196.824679] nix-daemon[851]: reaped child process 1880, status = succeeded6557worker1 # [ 196.828227] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506558worker1 # [ 196.841997] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506559worker1 # [ 196.853056] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6560worker1 # [ 196.977282] nix-daemon[851]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)6561worker1 # [ 196.996826] nix-daemon[851]: reaped child process 1910, status = succeeded6562lb # [ 196.941511] niks3-server[1146]: 2026/09/23 10:37:30 INFO Received uploads request method=POST path=/api/pending_closures6563worker1 # [ 197.013274] nix-grpc-daemon[1573]: time=2026-09-23T10:37:30.658Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6564worker1 # [ 197.014839] nix-grpc-daemon[1573]: time=2026-09-23T10:37:30.658Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6565lb # [ 196.969075] niks3-server[1146]: 2026/09/23 10:37:30 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6566worker1 # [ 197.043053] nix-grpc-daemon[1573]: time=2026-09-23T10:37:30.688Z level=INFO msg="Uploading 1 narinfos"6567lb # [ 196.982231] niks3-server[1146]: 2026/09/23 10:37:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6568lb # [ 196.992527] niks3-server[1146]: 2026/09/23 10:37:30 INFO Signed narinfos id=74 count=16569lb # [ 196.998602] niks3-server[1146]: 2026/09/23 10:37:30 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6570worker1 # [ 197.068614] nix-grpc-daemon[1573]: time=2026-09-23T10:37:30.713Z level=INFO msg="Upload complete. (211ms)"6571lb # [ 197.009801] niks3-server[1146]: 2026/09/23 10:37:30 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6572worker1 # [ 197.076235] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=5126573lb # [ 197.015725] niks3-server[1146]: 2026/09/23 10:37:30 INFO Completed upload id=746574lb # [ 197.019145] niks3-server[1146]: 2026/09/23 10:37:30 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6575worker1 # [ 197.084261] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506576worker1 # [ 197.096791] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6577worker1 # [ 197.099629] nix-daemon[851]: reaped child process 1905, status = succeeded6578worker1 # [ 197.111773] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6579worker1 # [ 198.373422] nix-daemon[851]: reaped child process 1914, status = succeeded6580lb # [ 198.361821] envoy[895]: conn peer=[2001:db8:1::1]:39716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9706 tx=5216 ms=54466581worker1 # [ 198.481717] nix-daemon[851]: accepted connection from pid 1930, user nix-grpc-daemon (trusted)6582worker1 # [ 198.499272] nix-daemon[851]: reaped child process 1933, status = succeeded6583lb # [ 198.443202] niks3-server[1146]: 2026/09/23 10:37:32 INFO Received uploads request method=POST path=/api/pending_closures6584worker1 # [ 198.516327] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6585worker1 # [ 198.517959] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.160Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6586worker1 # [ 198.544277] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.189Z level=INFO msg="Uploading 1 narinfos"6587lb # [ 198.481718] niks3-server[1146]: 2026/09/23 10:37:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6588lb # [ 198.496638] niks3-server[1146]: 2026/09/23 10:37:32 INFO Signed narinfos id=75 count=16589lb # [ 198.499277] niks3-server[1146]: 2026/09/23 10:37:32 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6590lb # [ 198.516510] niks3-server[1146]: 2026/09/23 10:37:32 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6591worker1 # [ 198.583622] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.228Z level=INFO msg="Upload complete. (262ms)"6592lb # [ 198.521643] niks3-server[1146]: 2026/09/23 10:37:32 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6593lb # [ 198.525307] niks3-server[1146]: 2026/09/23 10:37:32 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6594lb # [ 198.529371] niks3-server[1146]: 2026/09/23 10:37:32 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6595lb # [ 198.533663] niks3-server[1146]: 2026/09/23 10:37:32 INFO Completed upload id=756596worker1 # [ 198.601827] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=16597worker1 # [ 198.620366] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:32Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16598worker1 # [ 198.629148] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506599worker1 # [ 198.635606] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:32Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06600worker1 # [ 198.641935] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506601worker1 # [ 198.647313] nix-daemon[851]: reaped child process 1912, status = succeeded6602worker1 # [ 198.649600] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.291Z level=INFO msg="All 1 paths already cached"6603worker1 # [ 198.664215] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506604worker1 # [ 198.670354] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6605worker1 # [ 198.792945] nix-daemon[851]: accepted connection from pid 1939, user nix-grpc-daemon (trusted)6606worker1 # [ 198.809419] nix-daemon[851]: reaped child process 1942, status = succeeded6607lb # [ 198.751465] niks3-server[1146]: 2026/09/23 10:37:32 INFO Received uploads request method=POST path=/api/pending_closures6608worker1 # [ 198.823906] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.468Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6609worker1 # [ 198.825542] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.468Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6610lb # [ 198.781804] niks3-server[1146]: 2026/09/23 10:37:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6611worker1 # [ 198.850413] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.495Z level=INFO msg="Uploading 1 narinfos"6612lb # [ 198.796311] niks3-server[1146]: 2026/09/23 10:37:32 INFO Signed narinfos id=76 count=16613lb # [ 198.799362] niks3-server[1146]: 2026/09/23 10:37:32 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6614lb # [ 198.816835] niks3-server[1146]: 2026/09/23 10:37:32 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6615worker1 # [ 198.878351] nix-grpc-daemon[1573]: time=2026-09-23T10:37:32.523Z level=INFO msg="Upload complete. (204ms)"6616lb # [ 198.823384] niks3-server[1146]: 2026/09/23 10:37:32 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6617worker1 # [ 198.888433] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=6246618lb # [ 198.832405] niks3-server[1146]: 2026/09/23 10:37:32 INFO Completed upload id=766619worker1 # [ 198.897370] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506620lb # [ 198.835110] niks3-server[1146]: 2026/09/23 10:37:32 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6621worker1 # [ 198.905874] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6622worker1 # [ 198.908838] nix-daemon[851]: reaped child process 1937, status = succeeded6623worker1 # [ 198.921371] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6624worker1 # [ 200.388330] nix-daemon[851]: reaped child process 1946, status = succeeded6625worker1 # [ 200.489686] nix-daemon[851]: accepted connection from pid 1962, user nix-grpc-daemon (trusted)6626worker1 # [ 200.506270] nix-daemon[851]: reaped child process 1965, status = succeeded6627lb # [ 200.453099] niks3-server[1146]: 2026/09/23 10:37:34 INFO Received uploads request method=POST path=/api/pending_closures6628worker1 # [ 200.524401] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.168Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6629worker1 # [ 200.525927] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.168Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6630lb # [ 200.494038] niks3-server[1146]: 2026/09/23 10:37:34 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6631worker1 # [ 200.567914] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.212Z level=INFO msg="Uploading 1 narinfos"6632lb # [ 200.509177] niks3-server[1146]: 2026/09/23 10:37:34 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6633lb # [ 200.520155] niks3-server[1146]: 2026/09/23 10:37:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6634lb # [ 200.523240] niks3-server[1146]: 2026/09/23 10:37:34 INFO Signed narinfos id=77 count=16635lb # [ 200.529929] niks3-server[1146]: 2026/09/23 10:37:34 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6636lb # [ 200.533596] niks3-server[1146]: 2026/09/23 10:37:34 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6637lb # [ 200.537660] niks3-server[1146]: 2026/09/23 10:37:34 INFO Completed upload id=776638lb # [ 200.540432] niks3-server[1146]: 2026/09/23 10:37:34 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6639worker1 # [ 200.591958] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.237Z level=INFO msg="Upload complete. (260ms)"6640worker1 # [ 200.609867] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=16641worker1 # [ 200.624541] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:34Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16642worker1 # [ 200.634486] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506643worker1 # [ 200.639575] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:34Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06644worker1 # [ 200.644691] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506645worker1 # [ 200.649244] nix-daemon[851]: reaped child process 1944, status = succeeded6646worker1 # [ 200.651075] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.293Z level=INFO msg="All 1 paths already cached"6647worker1 # [ 200.664188] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506648worker1 # [ 200.667244] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6649worker1 # [ 200.791545] nix-daemon[851]: accepted connection from pid 1971, user nix-grpc-daemon (trusted)6650worker1 # [ 200.807043] nix-daemon[851]: reaped child process 1974, status = succeeded6651lb # [ 200.751137] niks3-server[1146]: 2026/09/23 10:37:34 INFO Received uploads request method=POST path=/api/pending_closures6652worker1 # [ 200.822072] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.466Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6653worker1 # [ 200.823521] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.466Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6654lb # [ 200.781885] niks3-server[1146]: 2026/09/23 10:37:34 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6655worker1 # [ 200.853287] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.498Z level=INFO msg="Uploading 1 narinfos"6656lb # [ 200.797767] niks3-server[1146]: 2026/09/23 10:37:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6657lb # [ 200.806463] niks3-server[1146]: 2026/09/23 10:37:34 INFO Signed narinfos id=78 count=16658worker1 # [ 200.876670] nix-grpc-daemon[1573]: time=2026-09-23T10:37:34.521Z level=INFO msg="Upload complete. (203ms)"6659lb # [ 200.814115] niks3-server[1146]: 2026/09/23 10:37:34 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6660lb # [ 200.822435] niks3-server[1146]: 2026/09/23 10:37:34 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6661worker1 # [ 200.884135] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=6246662lb # [ 200.826290] niks3-server[1146]: 2026/09/23 10:37:34 INFO Completed upload id=786663lb # [ 200.828821] niks3-server[1146]: 2026/09/23 10:37:34 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6664worker1 # [ 200.898822] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506665worker1 # [ 200.902239] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6666worker1 # [ 200.908179] nix-daemon[851]: reaped child process 1969, status = succeeded6667worker1 # [ 200.918445] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6668worker1 # [ 201.992973] nix-daemon[851]: reaped child process 1978, status = succeeded6669worker1 # [ 202.098274] nix-daemon[851]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6670worker1 # [ 202.116340] nix-daemon[851]: reaped child process 1997, status = succeeded6671lb # [ 202.060659] niks3-server[1146]: 2026/09/23 10:37:35 INFO Received uploads request method=POST path=/api/pending_closures6672worker1 # [ 202.133881] nix-grpc-daemon[1573]: time=2026-09-23T10:37:35.777Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6673worker1 # [ 202.136175] nix-grpc-daemon[1573]: time=2026-09-23T10:37:35.777Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6674lb # [ 202.094882] niks3-server[1146]: 2026/09/23 10:37:35 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6675worker1 # [ 202.174049] nix-grpc-daemon[1573]: time=2026-09-23T10:37:35.818Z level=INFO msg="Uploading 1 narinfos"6676lb # [ 202.111697] niks3-server[1146]: 2026/09/23 10:37:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6677lb # [ 202.122269] niks3-server[1146]: 2026/09/23 10:37:35 INFO Signed narinfos id=79 count=16678lb # [ 202.127786] niks3-server[1146]: 2026/09/23 10:37:35 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6679lb # [ 202.135872] niks3-server[1146]: 2026/09/23 10:37:35 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6680lb # [ 202.144781] niks3-server[1146]: 2026/09/23 10:37:35 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6681worker1 # [ 202.206658] nix-grpc-daemon[1573]: time=2026-09-23T10:37:35.851Z level=INFO msg="Upload complete. (279ms)"6682lb # [ 202.148480] niks3-server[1146]: 2026/09/23 10:37:35 INFO Completed upload id=796683lb # [ 202.152566] niks3-server[1146]: 2026/09/23 10:37:35 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6684worker1 # [ 202.225699] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=16685worker1 # [ 202.243818] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:35Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16686worker1 # [ 202.250884] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506687worker1 # [ 202.258036] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:35Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06688worker1 # [ 202.264841] nix-daemon[851]: reaped child process 1976, status = succeeded6689worker1 # [ 202.268982] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506690worker1 # [ 202.274559] nix-grpc-daemon[1573]: time=2026-09-23T10:37:35.918Z level=INFO msg="All 1 paths already cached"6691worker1 # [ 202.278553] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506692worker1 # [ 202.291728] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6693worker1 # [ 202.423703] nix-daemon[851]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)6694worker1 # [ 202.442579] nix-daemon[851]: reaped child process 2006, status = succeeded6695lb # [ 202.386140] niks3-server[1146]: 2026/09/23 10:37:36 INFO Received uploads request method=POST path=/api/pending_closures6696worker1 # [ 202.457184] nix-grpc-daemon[1573]: time=2026-09-23T10:37:36.102Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6697worker1 # [ 202.458647] nix-grpc-daemon[1573]: time=2026-09-23T10:37:36.102Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6698lb # [ 202.415913] niks3-server[1146]: 2026/09/23 10:37:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6699worker1 # [ 202.485785] nix-grpc-daemon[1573]: time=2026-09-23T10:37:36.130Z level=INFO msg="Uploading 1 narinfos"6700lb # [ 202.429506] niks3-server[1146]: 2026/09/23 10:37:36 INFO Signed narinfos id=80 count=16701lb # [ 202.438335] niks3-server[1146]: 2026/09/23 10:37:36 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6702lb # [ 202.450274] niks3-server[1146]: 2026/09/23 10:37:36 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6703worker1 # [ 202.516657] nix-grpc-daemon[1573]: time=2026-09-23T10:37:36.161Z level=INFO msg="Upload complete. (215ms)"6704lb # [ 202.456865] niks3-server[1146]: 2026/09/23 10:37:36 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6705lb # [ 202.462167] niks3-server[1146]: 2026/09/23 10:37:36 INFO Completed upload id=806706worker1 # [ 202.524093] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=6246707lb # [ 202.465104] niks3-server[1146]: 2026/09/23 10:37:36 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6708worker1 # [ 202.536902] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506709worker1 # [ 202.542807] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6710worker1 # [ 202.547403] nix-daemon[851]: reaped child process 2001, status = succeeded6711worker1 # [ 202.557751] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6712worker1 # [ 203.594727] nix-daemon[851]: reaped child process 2010, status = succeeded6713worker1 # [ 203.682724] nix-daemon[851]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6714worker1 # [ 203.700628] nix-daemon[851]: reaped child process 2029, status = succeeded6715lb # [ 203.637320] niks3-server[1146]: 2026/09/23 10:37:37 INFO Received uploads request method=POST path=/api/pending_closures6716worker1 # [ 203.712143] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.353Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6717worker1 # [ 203.713569] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.353Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6718lb # [ 203.675362] niks3-server[1146]: 2026/09/23 10:37:37 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6719lb # [ 203.687967] niks3-server[1146]: 2026/09/23 10:37:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6720worker1 # [ 203.756468] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.401Z level=INFO msg="Uploading 1 narinfos"6721lb # [ 203.700832] niks3-server[1146]: 2026/09/23 10:37:37 INFO Signed narinfos id=81 count=16722lb # [ 203.706394] niks3-server[1146]: 2026/09/23 10:37:37 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6723worker1 # [ 203.779884] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.425Z level=INFO msg="Upload complete. (211ms)"6724lb # [ 203.717936] niks3-server[1146]: 2026/09/23 10:37:37 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6725lb # [ 203.726168] niks3-server[1146]: 2026/09/23 10:37:37 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete6726lb # [ 203.731668] niks3-server[1146]: 2026/09/23 10:37:37 INFO Completed upload id=816727worker1 # [ 203.792799] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=16728lb # [ 203.734263] niks3-server[1146]: 2026/09/23 10:37:37 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6729worker1 # [ 203.807598] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:37Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16730worker1 # [ 203.813992] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506731worker1 # [ 203.819114] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:37Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06732worker1 # [ 203.824559] nix-daemon[851]: reaped child process 2008, status = succeeded6733worker1 # [ 203.826448] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506734worker1 # [ 203.833581] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.478Z level=INFO msg="All 2 paths already cached"6735worker1 # [ 203.841449] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506736worker1 # [ 203.850487] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6737worker1 # [ 203.973330] nix-daemon[851]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)6738worker1 # [ 203.989740] nix-daemon[851]: reaped child process 2038, status = succeeded6739lb # [ 203.933998] niks3-server[1146]: 2026/09/23 10:37:37 INFO Received uploads request method=POST path=/api/pending_closures6740worker1 # [ 204.005358] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.650Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6741worker1 # [ 204.006804] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.650Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6742lb # [ 203.964778] niks3-server[1146]: 2026/09/23 10:37:37 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6743worker1 # [ 204.037687] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.682Z level=INFO msg="Uploading 1 narinfos"6744lb # [ 203.977574] niks3-server[1146]: 2026/09/23 10:37:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign6745lb # [ 203.987394] niks3-server[1146]: 2026/09/23 10:37:37 INFO Signed narinfos id=82 count=16746lb # [ 203.995353] niks3-server[1146]: 2026/09/23 10:37:37 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6747worker1 # [ 204.064822] nix-grpc-daemon[1573]: time=2026-09-23T10:37:37.709Z level=INFO msg="Upload complete. (204ms)"6748lb # [ 204.003898] niks3-server[1146]: 2026/09/23 10:37:37 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete6749lb # [ 204.008647] niks3-server[1146]: 2026/09/23 10:37:37 INFO Completed upload id=826750worker1 # [ 204.072199] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=7526751lb # [ 204.012239] niks3-server[1146]: 2026/09/23 10:37:37 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6752worker1 # [ 204.085463] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506753worker1 # [ 204.091170] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6754worker1 # [ 204.095763] nix-daemon[851]: reaped child process 2033, status = succeeded6755worker1 # [ 204.106275] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6756worker1 # [ 204.976223] nix-daemon[851]: reaped child process 2042, status = succeeded6757worker1 # [ 205.072263] nix-daemon[851]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)6758worker1 # [ 205.088894] nix-daemon[851]: reaped child process 2061, status = succeeded6759worker1 # [ 205.104400] nix-grpc-daemon[1573]: time=2026-09-23T10:37:38.749Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6760worker1 # [ 205.105954] nix-grpc-daemon[1573]: time=2026-09-23T10:37:38.749Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6761lb # [ 205.031916] niks3-server[1146]: 2026/09/23 10:37:38 INFO Received uploads request method=POST path=/api/pending_closures6762lb # [ 205.077436] niks3-server[1146]: 2026/09/23 10:37:38 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6763lb # [ 205.087962] niks3-server[1146]: 2026/09/23 10:37:38 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6764worker1 # [ 205.159387] nix-grpc-daemon[1573]: time=2026-09-23T10:37:38.804Z level=INFO msg="Uploading 1 narinfos"6765lb # [ 205.099572] niks3-server[1146]: 2026/09/23 10:37:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign6766lb # [ 205.110459] niks3-server[1146]: 2026/09/23 10:37:38 INFO Signed narinfos id=83 count=16767lb # [ 205.119928] niks3-server[1146]: 2026/09/23 10:37:38 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6768lb # [ 205.128923] niks3-server[1146]: 2026/09/23 10:37:38 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete6769worker1 # [ 205.181542] nix-grpc-daemon[1573]: time=2026-09-23T10:37:38.826Z level=INFO msg="Upload complete. (264ms)"6770lb # [ 205.133292] niks3-server[1146]: 2026/09/23 10:37:38 INFO Completed upload id=836771lb # [ 205.135185] niks3-server[1146]: 2026/09/23 10:37:38 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6772worker1 # [ 205.197672] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=16773worker1 # [ 205.212300] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:38Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16774worker1 # [ 205.218634] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506775worker1 # [ 205.224308] nix-daemon[851]: reaped child process 2040, status = succeeded6776worker1 # [ 205.227521] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506777worker1 # [ 205.238620] nix-grpc-daemon[1573]: time=2026-09-23T10:37:38.883Z level=INFO msg="All 2 paths already cached"6778worker1 # [ 205.246680] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506779worker1 # [ 205.259014] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6780worker1 # [ 205.391937] nix-daemon[851]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)6781worker1 # [ 205.409503] nix-daemon[851]: reaped child process 2068, status = succeeded6782lb # [ 205.351997] niks3-server[1146]: 2026/09/23 10:37:39 INFO Received uploads request method=POST path=/api/pending_closures6783worker1 # [ 205.422907] nix-grpc-daemon[1573]: time=2026-09-23T10:37:39.067Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6784worker1 # [ 205.424585] nix-grpc-daemon[1573]: time=2026-09-23T10:37:39.067Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6785lb # [ 205.380118] niks3-server[1146]: 2026/09/23 10:37:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign6786worker1 # [ 205.448202] nix-grpc-daemon[1573]: time=2026-09-23T10:37:39.093Z level=INFO msg="Uploading 1 narinfos"6787lb # [ 205.392542] niks3-server[1146]: 2026/09/23 10:37:39 INFO Signed narinfos id=84 count=16788lb # [ 205.395791] niks3-server[1146]: 2026/09/23 10:37:39 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6789lb # [ 205.414943] niks3-server[1146]: 2026/09/23 10:37:39 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete6790worker1 # [ 205.483122] nix-grpc-daemon[1573]: time=2026-09-23T10:37:39.126Z level=INFO msg="Upload complete. (211ms)"6791lb # [ 205.422220] niks3-server[1146]: 2026/09/23 10:37:39 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6792lb # [ 205.428368] niks3-server[1146]: 2026/09/23 10:37:39 INFO Completed upload id=846793worker1 # [ 205.489736] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=7526794lb # [ 205.430979] niks3-server[1146]: 2026/09/23 10:37:39 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6795worker1 # [ 205.502211] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506796worker1 # [ 205.511203] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6797worker1 # [ 205.514592] nix-daemon[851]: reaped child process 2063, status = succeeded6798worker1 # [ 205.525281] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6799worker1 # [ 206.792805] nix-daemon[851]: reaped child process 2072, status = succeeded6800worker1 # [ 206.903429] nix-daemon[851]: accepted connection from pid 2088, user nix-grpc-daemon (trusted)6801worker1 # [ 206.922855] nix-daemon[851]: reaped child process 2091, status = succeeded6802lb # [ 206.859447] niks3-server[1146]: 2026/09/23 10:37:40 INFO Received uploads request method=POST path=/api/pending_closures6803worker1 # [ 206.932436] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.576Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6804worker1 # [ 206.933855] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.576Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6805lb # [ 206.898407] niks3-server[1146]: 2026/09/23 10:37:40 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6806worker1 # [ 206.986573] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.631Z level=INFO msg="Uploading 1 narinfos"6807lb # [ 206.918278] niks3-server[1146]: 2026/09/23 10:37:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign6808lb # [ 206.929390] niks3-server[1146]: 2026/09/23 10:37:40 INFO Signed narinfos id=85 count=16809lb # [ 206.935802] niks3-server[1146]: 2026/09/23 10:37:40 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6810lb # [ 206.949162] niks3-server[1146]: 2026/09/23 10:37:40 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6811lb # [ 206.956331] niks3-server[1146]: 2026/09/23 10:37:40 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete6812lb # [ 206.962559] niks3-server[1146]: 2026/09/23 10:37:40 INFO Completed upload id=856813worker1 # [ 207.017910] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.663Z level=INFO msg="Upload complete. (283ms)"6814lb # [ 206.968003] niks3-server[1146]: 2026/09/23 10:37:40 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6815worker1 # [ 207.031903] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=16816worker1 # [ 207.051776] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:40Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16817worker1 # [ 207.058188] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506818worker1 # [ 207.063593] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581506819worker1 # [ 207.068833] nix-daemon[851]: reaped child process 2070, status = succeeded6820worker1 # [ 207.071620] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.716Z level=INFO msg="All 2 paths already cached"6821worker1 # [ 207.079752] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581506822worker1 # [ 207.090741] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6823worker1 # [ 207.218937] nix-daemon[851]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)6824worker1 # [ 207.236518] nix-daemon[851]: reaped child process 2098, status = succeeded6825lb # [ 207.179120] niks3-server[1146]: 2026/09/23 10:37:40 INFO Received uploads request method=POST path=/api/pending_closures6826worker1 # [ 207.250316] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.895Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6827worker1 # [ 207.251786] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.895Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6828lb # [ 207.214432] niks3-server[1146]: 2026/09/23 10:37:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign6829worker1 # [ 207.282618] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.927Z level=INFO msg="Uploading 1 narinfos"6830lb # [ 207.227393] niks3-server[1146]: 2026/09/23 10:37:40 INFO Signed narinfos id=86 count=16831lb # [ 207.234301] niks3-server[1146]: 2026/09/23 10:37:40 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6832worker1 # [ 207.307957] nix-grpc-daemon[1573]: time=2026-09-23T10:37:40.953Z level=INFO msg="Upload complete. (206ms)"6833lb # [ 207.247850] niks3-server[1146]: 2026/09/23 10:37:40 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6834worker1 # [ 207.316310] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=7526835lb # [ 207.254225] niks3-server[1146]: 2026/09/23 10:37:40 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete6836lb # [ 207.259230] niks3-server[1146]: 2026/09/23 10:37:40 INFO Completed upload id=866837worker1 # [ 207.324903] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581506838lb # [ 207.262587] niks3-server[1146]: 2026/09/23 10:37:40 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6839worker1 # [ 207.334988] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6840worker1 # [ 207.339587] nix-daemon[851]: reaped child process 2093, status = succeeded6841worker1 # [ 207.349880] nix-daemon[851]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6842worker1 # [ 208.582541] nix-daemon[851]: reaped child process 2102, status = succeeded6843worker1 # [ 208.673588] nix-daemon[851]: accepted connection from pid 2118, user nix-grpc-daemon (trusted)6844worker1 # [ 208.692191] nix-daemon[851]: reaped child process 2121, status = succeeded6845lb # [ 208.634951] niks3-server[1146]: 2026/09/23 10:37:42 INFO Received uploads request method=POST path=/api/pending_closures6846worker1 # [ 208.708337] nix-grpc-daemon[1573]: time=2026-09-23T10:37:42.352Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6847worker1 # [ 208.709774] nix-grpc-daemon[1573]: time=2026-09-23T10:37:42.352Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6848lb # [ 208.676819] niks3-server[1146]: 2026/09/23 10:37:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign6849worker1 # [ 208.747453] nix-grpc-daemon[1573]: time=2026-09-23T10:37:42.392Z level=INFO msg="Uploading 1 narinfos"6850lb # [ 208.691193] niks3-server[1146]: 2026/09/23 10:37:42 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6851lb # [ 208.699451] niks3-server[1146]: 2026/09/23 10:37:42 INFO Signed narinfos id=87 count=16852lb # [ 208.707600] niks3-server[1146]: 2026/09/23 10:37:42 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6853worker1 # [ 208.775452] nix-grpc-daemon[1573]: time=2026-09-23T10:37:42.420Z level=INFO msg="Upload complete. (215ms)"6854lb # [ 208.715354] niks3-server[1146]: 2026/09/23 10:37:42 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6855lb # [ 208.721440] niks3-server[1146]: 2026/09/23 10:37:42 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete6856worker1 # [ 208.784646] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=16857lb # [ 208.726154] niks3-server[1146]: 2026/09/23 10:37:42 INFO Completed upload id=876858lb # [ 208.729521] niks3-server[1146]: 2026/09/23 10:37:42 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6859worker1 # [ 208.800214] nix-daemon[851]: reaped child process 2100, status = succeeded6860client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.69 seconds)6861worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6862worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6863client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6864client # [ 208.927909] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6865client # Running as unit: gpu.service; invocation ID: 2635fcf6ac1e4207abb6b39cb66d29fb6866client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6867lb: must succeed: curl -sf localhost:9901/clusters6868lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6869lb: must succeed: curl -sf localhost:9901/clusters6870lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6871worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6872worker1 # [ 209.192290] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581506873client # [ 209.136169] nix[1616]: don't know how to build these paths:6874client # [ 209.137308] nix[1616]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6875worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6876worker2 # [ 209.145826] nix-grpc-daemon[1992]: ts=2026-09-23T10:37:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401666877worker1 # [ 209.231054] nix-grpc-daemon[1473]: ts=2026-09-23T10:37:42Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06878client # [ 209.173819] nix[1616]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6879lb: must succeed: curl -sf localhost:9901/clusters6880lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6881lb: must succeed: curl -sf localhost:9901/clusters6882lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6883worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6884worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6885client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6886lb # [ 218.367451] envoy[895]: conn peer=[2001:db8:1::1]:33736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27127 tx=12354 ms=164226887client # [ 239.180741] nix[1616]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6888client # [ 239.193815] nix[1616]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6889client # [ 239.233408] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6890client # [ 239.241563] systemd[1]: gpu.service: Failed with result 'exit-code'.6891client # [ 239.245168] systemd[1]: gpu.service: Consumed 199ms CPU time over 30.310s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6892client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.05 seconds)6893client: must succeed: journalctl -u gpu -o cat6894client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6895lb: must succeed: curl -sf localhost:9901/clusters6896lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6897lb: must succeed: curl -sf localhost:9901/clusters6898lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6899worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6900worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6901(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.46 seconds)6902subtest: low disk drains a worker and builds go to the other6903worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6904worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.16 seconds)6905worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6906worker1 # [ 241.620001] nix-grpc-daemon[1473]: ts=2026-09-23T10:38:15Z level=info event=unhealthy reason=min_free6907worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6908client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26909client # don't know how to build these paths:6910client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6911worker2 # [ 242.051861] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401666912worker2 # [ 242.059970] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401666913worker1 # [ 242.145312] nix-grpc-daemon[1473]: ts=2026-09-23T10:38:15Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06914worker2 # [ 242.085874] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401666915worker1 # [ 242.151635] nix-grpc-daemon[1473]: ts=2026-09-23T10:38:15Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16916worker2 # [ 242.100764] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401666917client # copying 1 paths to 'lb:50051'...6918client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6919worker2 # [ 242.115896] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401666920worker2 # [ 242.131025] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6921worker2 # [ 242.286169] nix-daemon[849]: accepted connection from pid 2147, user nix-grpc-daemon (trusted)6922worker2 # [ 242.305680] nix-daemon[849]: reaped child process 2150, status = succeeded6923lb # [ 242.311697] niks3-server[1146]: 2026/09/23 10:38:15 INFO Received uploads request method=POST path=/api/pending_closures6924worker2 # [ 242.323349] nix-grpc-daemon[2052]: time=2026-09-23T10:38:15.971Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6925worker2 # [ 242.325888] nix-grpc-daemon[2052]: time=2026-09-23T10:38:15.973Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6926worker2 # [ 242.358875] nix-grpc-daemon[2052]: time=2026-09-23T10:38:16.006Z level=INFO msg="Uploading 1 narinfos"6927lb # [ 242.348631] niks3-server[1146]: 2026/09/23 10:38:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign6928lb # [ 242.364573] niks3-server[1146]: 2026/09/23 10:38:16 INFO Signed narinfos id=88 count=16929lb # [ 242.371078] niks3-server[1146]: 2026/09/23 10:38:16 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6930lb # [ 242.382266] niks3-server[1146]: 2026/09/23 10:38:16 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6931lb # [ 242.401575] niks3-server[1146]: 2026/09/23 10:38:16 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete6932lb # [ 242.407842] niks3-server[1146]: 2026/09/23 10:38:16 INFO Completed upload id=886933worker2 # [ 242.407796] nix-grpc-daemon[2052]: time=2026-09-23T10:38:16.055Z level=INFO msg="Upload complete. (259ms)"6934worker2 # [ 242.420373] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=0 paths=1 nar_bytes_in=5126935lb # [ 242.425999] niks3-server[1146]: 2026/09/23 10:38:16 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6936worker2 # [ 242.438023] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401666937worker2 # [ 242.448390] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6938worker2 # [ 242.460111] nix-daemon[849]: reaped child process 2145, status = succeeded6939client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6940client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6941worker2 # [ 242.469699] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6942client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6943client # farm-a> LOG-farm-a-drain6944worker2 # [ 243.935084] nix-daemon[849]: reaped child process 2154, status = succeeded6945worker2 # [ 244.063651] nix-daemon[849]: accepted connection from pid 2170, user nix-grpc-daemon (trusted)6946worker2 # [ 244.079271] nix-daemon[849]: reaped child process 2173, status = succeeded6947lb # [ 244.087276] niks3-server[1146]: 2026/09/23 10:38:17 INFO Received uploads request method=POST path=/api/pending_closures6948worker2 # [ 244.100437] nix-grpc-daemon[2052]: time=2026-09-23T10:38:17.746Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6949worker2 # [ 244.102439] nix-grpc-daemon[2052]: time=2026-09-23T10:38:17.746Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6950lb # [ 244.134557] niks3-server[1146]: 2026/09/23 10:38:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign6951worker2 # [ 244.144093] nix-grpc-daemon[2052]: time=2026-09-23T10:38:17.791Z level=INFO msg="Uploading 1 narinfos"6952lb # [ 244.147375] niks3-server[1146]: 2026/09/23 10:38:17 INFO Signed narinfos id=89 count=16953lb # [ 244.155365] niks3-server[1146]: 2026/09/23 10:38:17 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6954lb # [ 244.162489] niks3-server[1146]: 2026/09/23 10:38:17 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6955lb # [ 244.172223] niks3-server[1146]: 2026/09/23 10:38:17 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete6956lb # [ 244.179292] niks3-server[1146]: 2026/09/23 10:38:17 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6957worker2 # [ 244.178616] nix-grpc-daemon[2052]: time=2026-09-23T10:38:17.826Z level=INFO msg="Upload complete. (295ms)"6958lb # [ 244.182293] niks3-server[1146]: 2026/09/23 10:38:17 INFO Completed upload id=896959lb # [ 244.183765] niks3-server[1146]: 2026/09/23 10:38:17 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6960worker1 # [ 244.255980] nix-grpc-daemon[1473]: ts=2026-09-23T10:38:17Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06961worker1 # [ 244.258907] nix-grpc-daemon[1473]: ts=2026-09-23T10:38:17Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16962worker2 # [ 244.194384] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=16963worker2 # [ 244.212608] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401666964worker2 # [ 244.220957] nix-daemon[849]: reaped child process 2152, status = succeeded6965worker2 # [ 244.228874] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401666966worker2 # [ 244.241151] nix-grpc-daemon[2052]: time=2026-09-23T10:38:17.889Z level=INFO msg="All 1 paths already cached"6967client # copying 1 paths to 'lb:50051'...6968client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6969worker2 # [ 244.251637] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401666970worker2 # [ 244.267546] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6971worker2 # [ 244.409134] nix-daemon[849]: accepted connection from pid 2177, user nix-grpc-daemon (trusted)6972worker2 # [ 244.429320] nix-daemon[849]: reaped child process 2180, status = succeeded6973lb # [ 244.434052] niks3-server[1146]: 2026/09/23 10:38:18 INFO Received uploads request method=POST path=/api/pending_closures6974worker2 # [ 244.440989] nix-grpc-daemon[2052]: time=2026-09-23T10:38:18.088Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6975worker2 # [ 244.442500] nix-grpc-daemon[2052]: time=2026-09-23T10:38:18.088Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6976worker2 # [ 244.456866] nix-grpc-daemon[2052]: time=2026-09-23T10:38:18.104Z level=INFO msg="Uploading 1 narinfos"6977lb # [ 244.456663] niks3-server[1146]: 2026/09/23 10:38:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign6978lb # [ 244.467113] niks3-server[1146]: 2026/09/23 10:38:18 INFO Signed narinfos id=90 count=16979lb # [ 244.472487] niks3-server[1146]: 2026/09/23 10:38:18 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6980lb # [ 244.478435] niks3-server[1146]: 2026/09/23 10:38:18 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6981worker2 # [ 244.485105] nix-grpc-daemon[2052]: time=2026-09-23T10:38:18.133Z level=INFO msg="Upload complete. (205ms)"6982lb # [ 244.485824] niks3-server[1146]: 2026/09/23 10:38:18 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete6983lb # [ 244.492408] niks3-server[1146]: 2026/09/23 10:38:18 INFO Completed upload id=906984worker2 # [ 244.491661] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=0 paths=1 nar_bytes_in=6326985lb # [ 244.495140] niks3-server[1146]: 2026/09/23 10:38:18 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6986worker2 # [ 244.505651] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401666987worker2 # [ 244.510929] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6988worker2 # [ 244.515278] nix-daemon[849]: reaped child process 2175, status = succeeded6989client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6990worker2 # [ 244.527326] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6991client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6992client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6993client # farm-b> LOG-farm-b-drain6994worker2 # [ 245.563567] nix-daemon[849]: reaped child process 2184, status = succeeded6995worker2 # [ 245.680380] nix-daemon[849]: accepted connection from pid 2200, user nix-grpc-daemon (trusted)6996worker2 # [ 245.697237] nix-daemon[849]: reaped child process 2203, status = succeeded6997lb # [ 245.703707] niks3-server[1146]: 2026/09/23 10:38:19 INFO Received uploads request method=POST path=/api/pending_closures6998worker2 # [ 245.715896] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.360Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6999worker2 # [ 245.717802] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.360Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7000lb # [ 245.747024] niks3-server[1146]: 2026/09/23 10:38:19 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7001worker2 # [ 245.757617] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.405Z level=INFO msg="Uploading 1 narinfos"7002lb # [ 245.759700] niks3-server[1146]: 2026/09/23 10:38:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7003lb # [ 245.769619] niks3-server[1146]: 2026/09/23 10:38:19 INFO Signed narinfos id=91 count=17004lb # [ 245.774841] niks3-server[1146]: 2026/09/23 10:38:19 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7005worker2 # [ 245.785329] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.433Z level=INFO msg="Upload complete. (248ms)"7006lb # [ 245.785391] niks3-server[1146]: 2026/09/23 10:38:19 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7007lb # [ 245.792207] niks3-server[1146]: 2026/09/23 10:38:19 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7008lb # [ 245.797834] niks3-server[1146]: 2026/09/23 10:38:19 INFO Completed upload id=917009worker2 # [ 245.797555] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=17010lb # [ 245.800512] niks3-server[1146]: 2026/09/23 10:38:19 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7011worker1 # [ 245.871458] nix-grpc-daemon[1473]: ts=2026-09-23T10:38:19Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07012worker2 # [ 245.813094] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401667013worker1 # [ 245.877373] nix-grpc-daemon[1473]: ts=2026-09-23T10:38:19Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17014worker2 # [ 245.821429] nix-daemon[849]: reaped child process 2182, status = succeeded7015worker2 # [ 245.828524] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401667016worker2 # [ 245.839666] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.487Z level=INFO msg="All 2 paths already cached"7017client # copying 1 paths to 'lb:50051'...7018client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7019worker2 # [ 245.848970] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401667020worker2 # [ 245.862374] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)7021worker2 # [ 246.005993] nix-daemon[849]: accepted connection from pid 2207, user nix-grpc-daemon (trusted)7022worker2 # [ 246.023964] nix-daemon[849]: reaped child process 2210, status = succeeded7023lb # [ 246.027940] niks3-server[1146]: 2026/09/23 10:38:19 INFO Received uploads request method=POST path=/api/pending_closures7024worker2 # [ 246.037429] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.685Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7025worker2 # [ 246.038894] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.685Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7026lb # [ 246.056999] niks3-server[1146]: 2026/09/23 10:38:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7027worker2 # [ 246.065466] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.713Z level=INFO msg="Uploading 1 narinfos"7028lb # [ 246.073181] niks3-server[1146]: 2026/09/23 10:38:19 INFO Signed narinfos id=92 count=17029lb # [ 246.079275] niks3-server[1146]: 2026/09/23 10:38:19 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7030lb # [ 246.091171] niks3-server[1146]: 2026/09/23 10:38:19 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7031worker2 # [ 246.095306] nix-grpc-daemon[2052]: time=2026-09-23T10:38:19.743Z level=INFO msg="Upload complete. (219ms)"7032lb # [ 246.097030] niks3-server[1146]: 2026/09/23 10:38:19 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7033lb # [ 246.104769] niks3-server[1146]: 2026/09/23 10:38:19 INFO Completed upload id=927034worker2 # [ 246.103977] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=0 paths=1 nar_bytes_in=7607035lb # [ 246.107818] niks3-server[1146]: 2026/09/23 10:38:19 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7036worker2 # [ 246.117360] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401667037worker2 # [ 246.125620] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)7038worker2 # [ 246.128282] nix-daemon[849]: reaped child process 2205, status = succeeded7039client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7040worker2 # [ 246.140188] nix-daemon[849]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)7041client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7042client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7043client # farm-top> LOG-farm-top-drain7044worker2 # [ 247.182093] nix-daemon[849]: reaped child process 2214, status = succeeded7045worker2 # [ 247.285072] nix-daemon[849]: accepted connection from pid 2230, user nix-grpc-daemon (trusted)7046worker2 # [ 247.301953] nix-daemon[849]: reaped child process 2233, status = succeeded7047lb # [ 247.308786] niks3-server[1146]: 2026/09/23 10:38:20 INFO Received uploads request method=POST path=/api/pending_closures7048worker2 # [ 247.320388] nix-grpc-daemon[2052]: time=2026-09-23T10:38:20.967Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7049worker2 # [ 247.321902] nix-grpc-daemon[2052]: time=2026-09-23T10:38:20.967Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7050lb # [ 247.350141] niks3-server[1146]: 2026/09/23 10:38:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7051worker2 # [ 247.358751] nix-grpc-daemon[2052]: time=2026-09-23T10:38:21.006Z level=INFO msg="Uploading 1 narinfos"7052lb # [ 247.364365] niks3-server[1146]: 2026/09/23 10:38:21 INFO Signed narinfos id=93 count=17053lb # [ 247.366895] niks3-server[1146]: 2026/09/23 10:38:21 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7054lb # [ 247.378822] niks3-server[1146]: 2026/09/23 10:38:21 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7055lb # [ 247.389140] niks3-server[1146]: 2026/09/23 10:38:21 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7056lb # [ 247.391636] niks3-server[1146]: 2026/09/23 10:38:21 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7057lb # [ 247.395706] niks3-server[1146]: 2026/09/23 10:38:21 INFO Completed upload id=937058worker2 # [ 247.395552] nix-grpc-daemon[2052]: time=2026-09-23T10:38:21.043Z level=INFO msg="Upload complete. (246ms)"7059lb # [ 247.398246] niks3-server[1146]: 2026/09/23 10:38:21 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7060worker2 # [ 247.409094] nix-grpc-daemon[1992]: ts=2026-09-23T10:38:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40166 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=17061worker2 # [ 247.423025] nix-daemon[849]: reaped child process 2212, status = succeeded7062client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.61 seconds)7063client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7064client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7065worker1: must succeed: rm /nix/.rw-store/fill7066worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7067worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7068lb # [ 248.373122] envoy[895]: conn peer=[2001:db8:1::1]:37520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300757069lb # [ 248.378037] envoy[895]: conn peer=[2001:db8:1::1]:41698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9528 tx=4369 ms=53697070worker1 # [ 248.623965] nix-grpc-daemon[1473]: ts=2026-09-23T10:38:22Z level=info event=healthy reason=min_free7071worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7072(finished: subtest: low disk drains a worker and builds go to the other, in 9.39 seconds)7073(finished: run the VM test script, in 249.86 seconds)7074test script finished in 250.12s7075cleanup7076kill QemuMachine (pid 50)7077client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7078client # [2026-09-23T10:38:22Z INFO virtiofsd] Client disconnected, shutting down7079client # [2026-09-23T10:38:22Z INFO virtiofsd] Client disconnected, shutting down7080client # [2026-09-23T10:38:22Z INFO virtiofsd] Client disconnected, shutting down7081kill QemuMachine (pid 49)7082lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7083lb # [2026-09-23T10:38:22Z INFO virtiofsd] Client disconnected, shutting down7084lb # [2026-09-23T10:38:22Z INFO virtiofsd] Client disconnected, shutting down7085lb # [2026-09-23T10:38:22Z INFO virtiofsd] Client disconnected, shutting down7086kill QemuMachine (pid 51)7087worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7088worker1 # [2026-09-23T10:38:23Z INFO virtiofsd] Client disconnected, shutting down7089worker1 # [2026-09-23T10:38:23Z INFO virtiofsd] Client disconnected, shutting down7090worker1 # [2026-09-23T10:38:23Z INFO virtiofsd] Client disconnected, shutting down7091kill QemuMachine (pid 52)7092worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7093worker2 # [2026-09-23T10:38:23Z INFO virtiofsd] Client disconnected, shutting down7094worker2 # [2026-09-23T10:38:23Z INFO virtiofsd] Client disconnected, shutting down7095worker2 # [2026-09-23T10:38:23Z INFO virtiofsd] Client disconnected, shutting down7096(finished: cleanup, in 0.99 seconds)7097additionally exposed symbols:7098 client, lb, worker1, worker2,7099 vlan1,7100 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh