vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #498
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.AzCLZ0kiGg', fmt=raw size=107374182414worker1 # Disk image does not exist, creating the virtualisation disk image...15client # mke2fs 1.47.4 (6-Mar-2025)16worker2 # Disk image does not exist, creating the virtualisation disk image...17worker1: QEMU running (pid 51)18worker2: QEMU running (pid 52)19client: QEMU running (pid 49)20lb: QEMU running (pid 50)21lb # Disk image does not exist, creating the virtualisation disk image...22client # Discarding device blocks: 0/262144 done23worker1 # Formatting '/build/vm-state-worker1/tmp.uXSHubhAWl', fmt=raw size=107374182424lb # Formatting '/build/vm-state-lb/tmp.3QTJ3DucK4', fmt=raw size=107374182425client # Creating filesystem with 262144 4k blocks and 65536 inodes26worker2 # Formatting '/build/vm-state-worker2/tmp.V7q0RoKfXV', fmt=raw size=107374182427client # Filesystem UUID: 4e6ad68d-b89d-469d-afb4-92eda8e9d64828(finished: start all VMs, in 0.51 seconds)29worker1 # mke2fs 1.47.4 (6-Mar-2025)30client # Superblock backups stored on blocks:31client: waiting for unit network-addresses-eth1.service32worker1 # Discarding device blocks: 0/262144 done33client # 32768, 98304, 163840, 22937634client: waiting for the VM to finish booting35worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes36client # 37lb # mke2fs 1.47.4 (6-Mar-2025)38worker2 # mke2fs 1.47.4 (6-Mar-2025)39worker1 # Filesystem UUID: 75baad90-4b44-400d-a1a7-28f02042214740client # Allocating group tables: 0/8 done41lb # Discarding device blocks: 0/262144 done42worker2 # Discarding device blocks: 0/262144 done43worker1 # Superblock backups stored on blocks:44client # Writing inode tables: 0/8 done45lb # Creating filesystem with 262144 4k blocks and 65536 inodes46worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes47worker1 # 32768, 98304, 163840, 22937648client # Creating journal (8192 blocks): done49lb # Filesystem UUID: 6dc2e474-bf72-4bbe-8c12-76791bf6563a50worker2 # Filesystem UUID: e8e83e01-3ddb-4842-962c-142450055c0d51worker1 # 52client # Writing superblocks and filesystem accounting information: 0/8 done53lb # Superblock backups stored on blocks:54worker2 # Superblock backups stored on blocks:55worker1 # Allocating group tables: 0/8 done56client # 57lb # 32768, 98304, 163840, 22937658worker2 # 32768, 98304, 163840, 22937659worker1 # Writing inode tables: 0/8 done60client # Virtualisation disk image created.61lb # 62worker2 # 63worker1 # Creating journal (8192 blocks): done64client # Starting virtiofs daemons...65lb # Allocating group tables: 0/8 done66worker2 # Allocating group tables: 0/8 done67worker1 # Writing superblocks and filesystem accounting information: 0/8 done68client # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69lb # Writing inode tables: 0/8 done70worker2 # Writing inode tables: 0/8 done71worker1 # 72client # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73lb # Creating journal (8192 blocks): done74worker2 # Creating journal (8192 blocks): done75worker1 # Virtualisation disk image created.76client # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...77lb # Writing superblocks and filesystem accounting information: 0/8 done78worker2 # Writing superblocks and filesystem accounting information: 0/8 done79worker1 # Starting virtiofs daemons...80client # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)81lb # 82worker2 # 83worker1 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84client # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether85lb # Virtualisation disk image created.86worker2 # Virtualisation disk image created.87worker2 # Starting virtiofs daemons...88client # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)89lb # Starting virtiofs daemons...90worker2 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91client # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker1 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93client # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...94worker1 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95worker2 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether96lb # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)97client # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...98worker1 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether99worker2 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100lb # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101client # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests102worker1 # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...103worker2 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104lb # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...105client # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests106worker1 # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...107worker2 # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...108lb # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)109client # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests110worker1 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)111worker2 # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...112worker1 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether113worker2 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)114worker1 # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...115worker2 # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether116worker1 # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests117worker2 # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...118worker1 # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests119worker2 # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests120worker1 # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests121worker2 # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests122lb # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123worker2 # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests124lb # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-22T13:45:36Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-22T13:45:36Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-22T13:45:36Z INFO virtiofsd] Client connected, servicing requests131lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133lb # [ 0.000000] KASLR enabled134lb # [ 0.000000] random: crng init done135lb # [ 0.000000] Machine model: linux,dummy-virt136lb # [ 0.000000] efi: UEFI not found.137lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140lb # [ 0.000000] Zone ranges:141lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142lb # [ 0.000000] DMA32 empty143lb # [ 0.000000] Normal empty144lb # [ 0.000000] Device empty145lb # [ 0.000000] Movable zone start for each node146lb # [ 0.000000] Early memory node ranges147lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150lb # [ 0.000000] psci: probing for conduit method from DT.151lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.152lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153lb # [ 0.000000] psci: Trusted OS migration not required154lb # [ 0.000000] psci: SMC Calling Convention v1.1155client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]156lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)157client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026158lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296159client # [ 0.000000] KASLR enabled160lb # [ 0.000000] Detected PIPT I-cache on CPU0161client # [ 0.000000] random: crng init done162client # [ 0.000000] Machine model: linux,dummy-virt163lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)164client # [ 0.000000] efi: UEFI not found.165lb # [ 0.000000] CPU features: detected: GICv3 CPU interface166client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT167lb # [ 0.000000] CPU features: detected: Spectre-v4168lb # [ 0.000000] CPU features: detected: Spectre-BHB169client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]170lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38171client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]172client # [ 0.000000] Zone ranges:173lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23174client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]175lb # [ 0.000000] alternatives: applying boot alternatives176client # [ 0.000000] DMA32 empty177client # [ 0.000000] Normal empty178client # [ 0.000000] Device empty179client # [ 0.000000] Movable zone start for each node180client # [ 0.000000] Early memory node ranges181client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]182client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]183lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test/init regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration console=ttyAMA0,115200n8 console=tty0184client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000185client # [ 0.000000] psci: probing for conduit method from DT.186client # [ 0.000000] psci: PSCIv1.3 detected in firmware.187lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration", will be passed to user space.188client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs189client # [ 0.000000] psci: Trusted OS migration not required190lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes191client # [ 0.000000] psci: SMC Calling Convention v1.1192lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)193client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)194lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)195client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296196lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB197client # [ 0.000000] Detected PIPT I-cache on CPU0198lb # [ 0.000000] software IO TLB: area num 1.199client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)200lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB201client # [ 0.000000] CPU features: detected: GICv3 CPU interface202lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)203client # [ 0.000000] CPU features: detected: Spectre-v4204lb # [ 0.000000] Fallback order for Node 0: 0205client # [ 0.000000] CPU features: detected: Spectre-BHB206client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38207lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216208lb # [ 0.000000] Policy zone: DMA209client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23210lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off211client # [ 0.000000] alternatives: applying boot alternatives212lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1213lb # [ 0.000000] allocated 3145728 bytes of page_ext214lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages215lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups216lb # [ 0.000000] rcu: Hierarchical RCU implementation.217lb # [ 0.000000] rcu: RCU event tracing is enabled.218client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/iqhwx62rg63lzg5kb4y02k0590xv3v5s-nixos-system-client-test/init regInfo=/nix/store/z1q80gszksk98l07c912h8m1i571f245-closure-info/registration console=ttyAMA0,115200n8 console=tty0219lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.220lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.221lb # [ 0.000000] Rude variant of Tasks RCU enabled.222client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/z1q80gszksk98l07c912h8m1i571f245-closure-info/registration", will be passed to user space.223lb # [ 0.000000] Tracing variant of Tasks RCU enabled.224client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes225lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.226client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)227lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1228client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)229lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.230client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB231lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.232client # [ 0.000000] software IO TLB: area num 1.233client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB234lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.235client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)236lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0237lb # [ 0.000000] GICv3: 256 SPIs implemented238client # [ 0.000000] Fallback order for Node 0: 0239lb # [ 0.000000] GICv3: 0 Extended SPIs implemented240client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216241lb # [ 0.000000] Root IRQ handler: gic_handle_irq242client # [ 0.000000] Policy zone: DMA243lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI244client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off245lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0246client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1247lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000248client # [ 0.000000] allocated 3145728 bytes of page_ext249lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]250client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages251client # [ 0.000000] ftrace: allocated 294 pages with 4 groups252lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)253client # [ 0.000000] rcu: Hierarchical RCU implementation.254client # [ 0.000000] rcu: RCU event tracing is enabled.255lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)256client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.257lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000258worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]259client # [ 0.000000] Trampoline variant of Tasks RCU enabled.260client # [ 0.000000] Rude variant of Tasks RCU enabled.261lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000262client # [ 0.000000] Tracing variant of Tasks RCU enabled.263lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.264worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026265worker2 # [ 0.000000] KASLR enabled266client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.267worker2 # [ 0.000000] random: crng init done268lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns269worker2 # [ 0.000000] Machine model: linux,dummy-virt270client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1271worker2 # [ 0.000000] efi: UEFI not found.272lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).273client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.274worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT275lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns276worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]277client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.278worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]279lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns280worker2 # [ 0.000000] Zone ranges:281client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.282lb # [ 0.000034] arm-pv: using stolen time PV283worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]284client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0285worker2 # [ 0.000000] DMA32 empty286worker2 # [ 0.000000] Normal empty287client # [ 0.000000] GICv3: 256 SPIs implemented288lb # [ 0.000672] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)289worker2 # [ 0.000000] Device empty290client # [ 0.000000] GICv3: 0 Extended SPIs implemented291lb # [ 0.000866] Console: colour dummy device 80x25292worker2 # [ 0.000000] Movable zone start for each node293client # [ 0.000000] Root IRQ handler: gic_handle_irq294lb # [ 0.000874] printk: legacy console [tty0] enabled295worker2 # [ 0.000000] Early memory node ranges296client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI297worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]298client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0299lb # [ 0.001072] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)300worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]301lb # [ 0.001078] pid_max: default: 32768 minimum: 301302client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000303worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000304client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]305lb # [ 0.001155] LSM: initializing lsm=capability,landlock,yama,bpf,ima306lb # [ 0.001329] landlock: Up and running.307worker2 # [ 0.000000] psci: probing for conduit method from DT.308worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]309lb # [ 0.001332] Yama: becoming mindful.310worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.311client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)312lb # [ 0.002001] LSM support for eBPF active313worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs314client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)315worker2 # [ 0.000000] psci: Trusted OS migration not required316lb # [ 0.002187] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)317worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026318worker2 # [ 0.000000] psci: SMC Calling Convention v1.1319worker1 # [ 0.000000] KASLR enabled320client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000321worker1 # [ 0.000000] random: crng init done322lb # [ 0.002220] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)323worker1 # [ 0.000000] Machine model: linux,dummy-virt324client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000325lb # [ 0.004171] rcu: Hierarchical SRCU implementation.326worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)327worker1 # [ 0.000000] efi: UEFI not found.328lb # [ 0.004176] rcu: Max phase no-delay instances is 1000.329client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.330worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296331lb # [ 0.005404] fsl-mc MSI: its@8080000 domain created332worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT333worker2 # [ 0.000000] Detected PIPT I-cache on CPU0334lb # [ 0.005503] EFI services will not be available.335client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns336worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]337worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)338lb # [ 0.005625] smp: Bringing up secondary CPUs ...339client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).340worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]341lb # [ 0.005634] smp: Brought up 1 node, 1 CPU342worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface343worker1 # [ 0.000000] Zone ranges:344lb # [ 0.005637] SMP: Total of 1 processors activated.345worker2 # [ 0.000000] CPU features: detected: Spectre-v4346lb # [ 0.005642] CPU: All CPU(s) started at EL1347worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]348client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns349worker2 # [ 0.000000] CPU features: detected: Spectre-BHB350worker1 # [ 0.000000] DMA32 empty351worker1 # [ 0.000000] Normal empty352lb # [ 0.005652] CPU features: detected: Branch Target Identification353worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38354worker1 # [ 0.000000] Device empty355client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns356lb # [ 0.005656] CPU features: detected: ARMv8.4 Translation Table Level357client # [ 0.000034] arm-pv: using stolen time PV358worker1 # [ 0.000000] Movable zone start for each node359worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23360worker1 # [ 0.000000] Early memory node ranges361worker2 # [ 0.000000] alternatives: applying boot alternatives362lb # [ 0.005659] CPU features: detected: Instruction cache invalidation not required for I/D coherence363client # [ 0.000662] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)364worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]365client # [ 0.000833] Console: colour dummy device 80x25366lb # [ 0.005663] CPU features: detected: Data cache clean to the PoU not required for I/D coherence367worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]368client # [ 0.000840] printk: legacy console [tty0] enabled369worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000370lb # [ 0.005667] CPU features: detected: Common not Private translations371worker1 # [ 0.000000] psci: probing for conduit method from DT.372lb # [ 0.005670] CPU features: detected: CRC32 instructions373client # [ 0.001038] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)374worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.375client # [ 0.001046] pid_max: default: 32768 minimum: 301376lb # [ 0.005673] CPU features: detected: Data cache clean to Point of Deep Persistence377worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/j9bwyfrqbibwii8xrbm0g0c2giw11pp5-nixos-system-worker2-test/init regInfo=/nix/store/gwdpf4z5n2mxxzzijz0w78mlvm1p1g1z-closure-info/registration console=ttyAMA0,115200n8 console=tty0378worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs379client # [ 0.001153] LSM: initializing lsm=capability,landlock,yama,bpf,ima380lb # [ 0.005676] CPU features: detected: Data cache clean to Point of Persistence381worker1 # [ 0.000000] psci: Trusted OS migration not required382client # [ 0.001313] landlock: Up and running.383worker1 # [ 0.000000] psci: SMC Calling Convention v1.1384client # [ 0.001316] Yama: becoming mindful.385lb # [ 0.005679] CPU features: detected: Data independent timing control (DIT)386client # [ 0.001873] LSM support for eBPF active387worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/gwdpf4z5n2mxxzzijz0w78mlvm1p1g1z-closure-info/registration", will be passed to user space.388lb # [ 0.005683] CPU features: detected: E0PD389worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)390client # [ 0.002020] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)391worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes392lb # [ 0.005685] CPU features: detected: Enhanced Counter Virtualization393worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296394worker1 # [ 0.000000] Detected PIPT I-cache on CPU0395client # [ 0.002053] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)396worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)397lb # [ 0.005688] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)398client # [ 0.003914] rcu: Hierarchical SRCU implementation.399lb # [ 0.005692] CPU features: detected: Enhanced Virtualization Traps400worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)401worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)402client # [ 0.003918] rcu: Max phase no-delay instances is 1000.403lb # [ 0.005695] CPU features: detected: Fine Grained Traps404client # [ 0.005200] fsl-mc MSI: its@8080000 domain created405worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB406worker2 # [ 0.000000] software IO TLB: area num 1.407worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface408client # [ 0.005293] EFI services will not be available.409lb # [ 0.005698] CPU features: detected: Generic authentication (architected QARMA5 algorithm)410worker1 # [ 0.000000] CPU features: detected: Spectre-v4411client # [ 0.005359] smp: Bringing up secondary CPUs ...412worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB413lb # [ 0.005703] CPU features: detected: RCpc load-acquire (LDAPR)414worker1 # [ 0.000000] CPU features: detected: Spectre-BHB415client # [ 0.005367] smp: Brought up 1 node, 1 CPU416lb # [ 0.005706] CPU features: detected: LSE atomic instructions417client # [ 0.005371] SMP: Total of 1 processors activated.418worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)419worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38420lb # [ 0.005709] CPU features: detected: Privileged Access Never421worker2 # [ 0.000000] Fallback order for Node 0: 0422client # [ 0.005376] CPU: All CPU(s) started at EL1423worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23424lb # [ 0.005711] CPU features: detected: PMUv3425client # [ 0.005386] CPU features: detected: Branch Target Identification426worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216427worker1 # [ 0.000000] alternatives: applying boot alternatives428worker2 # [ 0.000000] Policy zone: DMA429lb # [ 0.005715] CPU features: detected: RAS Extension Support430client # [ 0.005390] CPU features: detected: ARMv8.4 Translation Table Level431lb # [ 0.005718] CPU features: detected: RASv1p1 Extension Support432worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off433lb # [ 0.005720] CPU features: detected: Random Number Generator434client # [ 0.005393] CPU features: detected: Instruction cache invalidation not required for I/D coherence435worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1436lb # [ 0.005723] CPU features: detected: Speculation barrier (SB)437worker2 # [ 0.000000] allocated 3145728 bytes of page_ext438client # [ 0.005397] CPU features: detected: Data cache clean to the PoU not required for I/D coherence439lb # [ 0.005725] CPU features: detected: Stage-2 Force Write-Back440client # [ 0.005401] CPU features: detected: Common not Private translations441lb # [ 0.005728] CPU features: detected: TLB range maintenance instructions442worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages443worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/v2qpsr299a5icg4s4hqkpgiyhjwjq5zh-nixos-system-worker1-test/init regInfo=/nix/store/vkn7l0bb2har9b7pkp16z3kw2kh26hpj-closure-info/registration console=ttyAMA0,115200n8 console=tty0444client # [ 0.005404] CPU features: detected: CRC32 instructions445worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups446lb # [ 0.005733] CPU features: detected: Speculative Store Bypassing Safe (SSBS)447client # [ 0.005407] CPU features: detected: Data cache clean to Point of Deep Persistence448worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.449lb # [ 0.005775] alternatives: applying system-wide alternatives450worker2 # [ 0.000000] rcu: RCU event tracing is enabled.451worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/vkn7l0bb2har9b7pkp16z3kw2kh26hpj-closure-info/registration", will be passed to user space.452client # [ 0.005410] CPU features: detected: Data cache clean to Point of Persistence453lb # [ 0.008683] CPU features: detected: BBM Level 2 without TLB conflict abort454worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.455worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes456client # [ 0.005413] CPU features: detected: Data independent timing control (DIT)457worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.458client # [ 0.005416] CPU features: detected: E0PD459worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)460worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.461lb # [ 0.008861] Memory: 1405652K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)462worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.463client # [ 0.005419] CPU features: detected: Enhanced Counter Virtualization464lb # [ 0.009221] devtmpfs: initialized465worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)466client # [ 0.005422] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)467worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.468lb # [ 0.011062] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)469worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB470client # [ 0.005426] CPU features: detected: Enhanced Virtualization Traps471worker1 # [ 0.000000] software IO TLB: area num 1.472worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1473lb # [ 0.011086] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).474client # [ 0.005428] CPU features: detected: Fine Grained Traps475worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB476worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.477lb # [ 0.011259] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL478client # [ 0.005431] CPU features: detected: Generic authentication (architected QARMA5 algorithm)479lb # [ 0.011263] 0 pages in range for non-PLT usage480worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)481client # [ 0.005436] CPU features: detected: RCpc load-acquire (LDAPR)482worker1 # [ 0.000000] Fallback order for Node 0: 0483lb # [ 0.011264] 508272 pages in range for PLT usage484worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.485lb # [ 0.011366] pinctrl core: initialized pinctrl subsystem486client # [ 0.005439] CPU features: detected: LSE atomic instructions487worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216488worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.489worker1 # [ 0.000000] Policy zone: DMA490lb # [ 0.012187] DMI not present or invalid.491client # [ 0.005442] CPU features: detected: Privileged Access Never492client # [ 0.005444] CPU features: detected: PMUv3493lb # [ 0.015866] NET: Registered PF_NETLINK/PF_ROUTE protocol family494worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0495worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off496worker2 # [ 0.000000] GICv3: 256 SPIs implemented497client # [ 0.005447] CPU features: detected: RAS Extension Support498lb # [ 0.018097] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations499worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented500worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1501client # [ 0.005450] CPU features: detected: RASv1p1 Extension Support502worker1 # [ 0.000000] allocated 3145728 bytes of page_ext503worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq504lb # [ 0.018269] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations505client # [ 0.005452] CPU features: detected: Random Number Generator506worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI507worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages508worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0509lb # [ 0.018466] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations510worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups511worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.512lb # [ 0.018488] audit: initializing netlink subsys (disabled)513worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000514worker1 # [ 0.000000] rcu: RCU event tracing is enabled.515worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]516lb # [ 0.019029] thermal_sys: Registered thermal governor 'fair_share'517worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.518lb # [ 0.019031] thermal_sys: Registered thermal governor 'bang_bang'519worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)520worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.521lb # [ 0.019035] thermal_sys: Registered thermal governor 'step_wise'522worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.523worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)524lb # [ 0.019037] thermal_sys: Registered thermal governor 'user_space'525worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.526worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000527worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.528lb # [ 0.019042] thermal_sys: Registered thermal governor 'power_allocator'529worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000530worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1531worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.532lb # [ 0.019067] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1533worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.534lb # [ 0.019076] cpuidle: using governor ladder535lb # [ 0.019082] cpuidle: using governor menu536worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns537worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.538worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).539lb # [ 0.019268] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.540client # [ 0.005455] CPU features: detected: Speculation barrier (SB)541lb # [ 0.019283] ASID allocator initialised with 65536 entries542worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.543client # [ 0.005457] CPU features: detected: Stage-2 Force Write-Back544lb # [ 0.020554] Serial: AMBA PL011 UART driver545worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns546worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0547worker1 # [ 0.000000] GICv3: 256 SPIs implemented548client # [ 0.005460] CPU features: detected: TLB range maintenance instructions549lb # [ 0.025979] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1550worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns551worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented552worker2 # [ 0.000031] arm-pv: using stolen time PV553client # [ 0.005465] CPU features: detected: Speculative Store Bypassing Safe (SSBS)554lb # [ 0.026103] printk: console [ttyAMA0] enabled555worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq556client # [ 0.005504] alternatives: applying system-wide alternatives557worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI558worker2 # [ 0.000465] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)559worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0560client # [ 0.008396] CPU features: detected: BBM Level 2 without TLB conflict abort561worker2 # [ 0.000652] Console: colour dummy device 80x25562worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000563worker2 # [ 0.000660] printk: legacy console [tty0] enabled564worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]565client # [ 0.008587] Memory: 1405624K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)566client # [ 0.009002] devtmpfs: initialized567worker2 # [ 0.000847] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)568worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)569worker2 # [ 0.000854] pid_max: default: 32768 minimum: 301570client # [ 0.010750] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)571worker2 # [ 0.000939] LSM: initializing lsm=capability,landlock,yama,bpf,ima572worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)573worker2 # [ 0.001084] landlock: Up and running.574client # [ 0.010773] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).575worker2 # [ 0.001087] Yama: becoming mindful.576worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000577client # [ 0.010965] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL578worker2 # [ 0.001602] LSM support for eBPF active579worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000580client # [ 0.010970] 0 pages in range for non-PLT usage581lb # [ 0.145232] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages582worker2 # [ 0.001754] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)583client # [ 0.010971] 508272 pages in range for PLT usage584worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.585lb # [ 0.145249] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page586client # [ 0.011080] pinctrl core: initialized pinctrl subsystem587worker2 # [ 0.001788] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)588client # [ 0.011826] DMI not present or invalid.589worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns590lb # [ 0.145254] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages591worker2 # [ 0.003567] rcu: Hierarchical SRCU implementation.592client # [ 0.015255] NET: Registered PF_NETLINK/PF_ROUTE protocol family593worker2 # [ 0.003572] rcu: Max phase no-delay instances is 1000.594worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).595lb # [ 0.145259] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page596worker2 # [ 0.004881] fsl-mc MSI: its@8080000 domain created597client # [ 0.017439] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations598lb # [ 0.145263] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages599worker2 # [ 0.004973] EFI services will not be available.600worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns601client # [ 0.017628] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations602worker2 # [ 0.005049] smp: Bringing up secondary CPUs ...603lb # [ 0.145268] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page604worker2 # [ 0.005058] smp: Brought up 1 node, 1 CPU605worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns606lb # [ 0.145272] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages607client # [ 0.017830] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations608worker2 # [ 0.005061] SMP: Total of 1 processors activated.609worker1 # [ 0.000029] arm-pv: using stolen time PV610client # [ 0.017852] audit: initializing netlink subsys (disabled)611lb # [ 0.145277] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page612worker2 # [ 0.005066] CPU: All CPU(s) started at EL1613client # [ 0.018446] thermal_sys: Registered thermal governor 'fair_share'614worker2 # [ 0.005077] CPU features: detected: Branch Target Identification615worker1 # [ 0.000397] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)616client # [ 0.018448] thermal_sys: Registered thermal governor 'bang_bang'617worker1 # [ 0.000572] Console: colour dummy device 80x25618worker2 # [ 0.005082] CPU features: detected: ARMv8.4 Translation Table Level619worker1 # [ 0.000579] printk: legacy console [tty0] enabled620client # [ 0.018452] thermal_sys: Registered thermal governor 'step_wise'621lb # [ 0.152455] fbcon: Taking over console622worker2 # [ 0.005085] CPU features: detected: Instruction cache invalidation not required for I/D coherence623client # [ 0.018455] thermal_sys: Registered thermal governor 'user_space'624lb # [ 0.152470] ACPI: Interpreter disabled.625worker1 # [ 0.000777] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)626worker2 # [ 0.005089] CPU features: detected: Data cache clean to the PoU not required for I/D coherence627client # [ 0.018460] thermal_sys: Registered thermal governor 'power_allocator'628worker1 # [ 0.000783] pid_max: default: 32768 minimum: 301629worker2 # [ 0.005092] CPU features: detected: Common not Private translations630client # [ 0.018485] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1631worker1 # [ 0.000862] LSM: initializing lsm=capability,landlock,yama,bpf,ima632worker2 # [ 0.005096] CPU features: detected: CRC32 instructions633client # [ 0.018493] cpuidle: using governor ladder634worker1 # [ 0.001030] landlock: Up and running.635worker1 # [ 0.001033] Yama: becoming mindful.636client # [ 0.018500] cpuidle: using governor menu637worker2 # [ 0.005099] CPU features: detected: Data cache clean to Point of Deep Persistence638worker1 # [ 0.001478] LSM support for eBPF active639client # [ 0.018686] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.640worker2 # [ 0.005102] CPU features: detected: Data cache clean to Point of Persistence641worker1 # [ 0.001614] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)642client # [ 0.018701] ASID allocator initialised with 65536 entries643client # [ 0.019970] Serial: AMBA PL011 UART driver644worker2 # [ 0.005105] CPU features: detected: Data independent timing control (DIT)645worker1 # [ 0.001647] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)646worker2 # [ 0.005108] CPU features: detected: E0PD647worker1 # [ 0.003486] rcu: Hierarchical SRCU implementation.648client # [ 0.025257] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1649worker2 # [ 0.005111] CPU features: detected: Enhanced Counter Virtualization650worker1 # [ 0.003491] rcu: Max phase no-delay instances is 1000.651client # [ 0.025409] printk: console [ttyAMA0] enabled652worker2 # [ 0.005114] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)653worker1 # [ 0.004746] fsl-mc MSI: its@8080000 domain created654worker1 # [ 0.004835] EFI services will not be available.655worker2 # [ 0.005118] CPU features: detected: Enhanced Virtualization Traps656client # [ 0.145591] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages657worker1 # [ 0.004899] smp: Bringing up secondary CPUs ...658worker2 # [ 0.005121] CPU features: detected: Fine Grained Traps659worker1 # [ 0.004907] smp: Brought up 1 node, 1 CPU660client # [ 0.145614] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page661lb # [ 0.161467] iommu: Default domain type: Translated662worker1 # [ 0.004910] SMP: Total of 1 processors activated.663worker2 # [ 0.005124] CPU features: detected: Generic authentication (architected QARMA5 algorithm)664client # [ 0.145620] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages665lb # [ 0.161478] iommu: DMA domain TLB invalidation policy: strict mode666worker1 # [ 0.004916] CPU: All CPU(s) started at EL1667worker2 # [ 0.005129] CPU features: detected: RCpc load-acquire (LDAPR)668lb # [ 0.161835] SCSI subsystem initialized669client # [ 0.145625] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page670worker1 # [ 0.004927] CPU features: detected: Branch Target Identification671worker2 # [ 0.005132] CPU features: detected: LSE atomic instructions672client # [ 0.145629] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages673worker1 # [ 0.004932] CPU features: detected: ARMv8.4 Translation Table Level674worker2 # [ 0.005135] CPU features: detected: Privileged Access Never675lb # [ 0.163759] usbcore: registered new interface driver usbfs676worker2 # [ 0.005138] CPU features: detected: PMUv3677client # [ 0.145634] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page678lb # [ 0.163790] usbcore: registered new interface driver hub679worker1 # [ 0.004935] CPU features: detected: Instruction cache invalidation not required for I/D coherence680worker2 # [ 0.005141] CPU features: detected: RAS Extension Support681client # [ 0.145639] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages682lb # [ 0.163806] usbcore: registered new device driver usb683worker2 # [ 0.005144] CPU features: detected: RASv1p1 Extension Support684worker1 # [ 0.004938] CPU features: detected: Data cache clean to the PoU not required for I/D coherence685client # [ 0.145643] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page686lb # [ 0.164090] pps_core: LinuxPPS API ver. 1 registered687worker2 # [ 0.005146] CPU features: detected: Random Number Generator688worker1 # [ 0.004942] CPU features: detected: Common not Private translations689lb # [ 0.164096] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>690worker2 # [ 0.005149] CPU features: detected: Speculation barrier (SB)691worker1 # [ 0.004945] CPU features: detected: CRC32 instructions692lb # [ 0.164114] PTP clock support registered693worker2 # [ 0.005152] CPU features: detected: Stage-2 Force Write-Back694lb # [ 0.164171] EDAC MC: Ver: 3.0.0695worker1 # [ 0.004948] CPU features: detected: Data cache clean to Point of Deep Persistence696client # [ 0.152917] fbcon: Taking over console697worker2 # [ 0.005155] CPU features: detected: TLB range maintenance instructions698lb # [ 0.168671] scmi_core: SCMI protocol bus registered699client # [ 0.152930] ACPI: Interpreter disabled.700worker1 # [ 0.004952] CPU features: detected: Data cache clean to Point of Persistence701worker2 # [ 0.005159] CPU features: detected: Speculative Store Bypassing Safe (SSBS)702client # [ 0.154703] iommu: Default domain type: Translated703worker1 # [ 0.004955] CPU features: detected: Data independent timing control (DIT)704lb # [ 0.169612] FPGA manager framework705worker2 # [ 0.005196] alternatives: applying system-wide alternatives706lb # [ 0.170519] vgaarb: loaded707worker1 # [ 0.004958] CPU features: detected: E0PD708client # [ 0.154713] iommu: DMA domain TLB invalidation policy: strict mode709worker2 # [ 0.008150] CPU features: detected: BBM Level 2 without TLB conflict abort710worker1 # [ 0.004960] CPU features: detected: Enhanced Counter Virtualization711client # [ 0.156351] SCSI subsystem initialized712lb # [ 0.171121] clocksource: Switched to clocksource arch_sys_counter713worker1 # [ 0.004964] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)714worker2 # [ 0.008415] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)715worker1 # [ 0.004967] CPU features: detected: Enhanced Virtualization Traps716worker2 # [ 0.008778] devtmpfs: initialized717worker1 # [ 0.004970] CPU features: detected: Fine Grained Traps718worker2 # [ 0.010590] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)719worker1 # [ 0.004974] CPU features: detected: Generic authentication (architected QARMA5 algorithm)720worker2 # [ 0.010612] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).721worker1 # [ 0.004978] CPU features: detected: RCpc load-acquire (LDAPR)722worker1 # [ 0.004981] CPU features: detected: LSE atomic instructions723worker2 # [ 0.010787] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL724worker2 # [ 0.010791] 0 pages in range for non-PLT usage725worker1 # [ 0.004984] CPU features: detected: Privileged Access Never726worker2 # [ 0.010792] 508272 pages in range for PLT usage727worker1 # [ 0.004986] CPU features: detected: PMUv3728lb # [ 0.177221] VFS: Disk quotas dquot_6.6.0729worker1 # [ 0.004989] CPU features: detected: RAS Extension Support730lb # [ 0.177262] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)731worker1 # [ 0.004992] CPU features: detected: RASv1p1 Extension Support732worker1 # [ 0.004995] CPU features: detected: Random Number Generator733lb # [ 0.178894] netfs: FS-Cache loaded734lb # [ 0.179010] pnp: PnP ACPI: disabled735client # [ 0.163973] usbcore: registered new interface driver usbfs736worker1 # [ 0.004997] CPU features: detected: Speculation barrier (SB)737client # [ 0.164005] usbcore: registered new interface driver hub738worker1 # [ 0.005000] CPU features: detected: Stage-2 Force Write-Back739client # [ 0.164021] usbcore: registered new device driver usb740client # [ 0.164303] pps_core: LinuxPPS API ver. 1 registered741client # [ 0.164309] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>742client # [ 0.164319] PTP clock support registered743lb # [ 0.182814] NET: Registered PF_INET protocol family744client # [ 0.164368] EDAC MC: Ver: 3.0.0745lb # [ 0.183070] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)746client # [ 0.168965] scmi_core: SCMI protocol bus registered747client # [ 0.169933] FPGA manager framework748worker2 # [ 0.010913] pinctrl core: initialized pinctrl subsystem749client # [ 0.170898] vgaarb: loaded750worker2 # [ 0.011660] DMI not present or invalid.751worker2 # [ 0.014858] NET: Registered PF_NETLINK/PF_ROUTE protocol family752client # [ 0.171523] clocksource: Switched to clocksource arch_sys_counter753worker2 # [ 0.017140] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations754client # [ 0.172116] VFS: Disk quotas dquot_6.6.0755worker2 # [ 0.017322] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations756client # [ 0.172147] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)757worker1 # [ 0.005003] CPU features: detected: TLB range maintenance instructions758worker2 # [ 0.017523] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations759client # [ 0.174574] netfs: FS-Cache loaded760client # [ 0.174679] pnp: PnP ACPI: disabled761worker1 # [ 0.005008] CPU features: detected: Speculative Store Bypassing Safe (SSBS)762worker2 # [ 0.017545] audit: initializing netlink subsys (disabled)763worker1 # [ 0.005044] alternatives: applying system-wide alternatives764worker2 # [ 0.018140] thermal_sys: Registered thermal governor 'fair_share'765worker1 # [ 0.007897] CPU features: detected: BBM Level 2 without TLB conflict abort766worker2 # [ 0.018142] thermal_sys: Registered thermal governor 'bang_bang'767worker2 # [ 0.018145] thermal_sys: Registered thermal governor 'step_wise'768worker2 # [ 0.018148] thermal_sys: Registered thermal governor 'user_space'769worker1 # [ 0.008091] Memory: 1405632K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)770worker1 # [ 0.008471] devtmpfs: initialized771worker2 # [ 0.018153] thermal_sys: Registered thermal governor 'power_allocator'772worker1 # [ 0.010223] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)773worker2 # [ 0.018179] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1774worker2 # [ 0.018186] cpuidle: using governor ladder775worker1 # [ 0.010252] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).776worker2 # [ 0.018192] cpuidle: using governor menu777worker1 # [ 0.010427] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL778worker2 # [ 0.018384] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.779worker1 # [ 0.010431] 0 pages in range for non-PLT usage780worker2 # [ 0.018399] ASID allocator initialised with 65536 entries781worker1 # [ 0.010432] 508272 pages in range for PLT usage782worker2 # [ 0.019614] Serial: AMBA PL011 UART driver783worker1 # [ 0.010535] pinctrl core: initialized pinctrl subsystem784worker1 # [ 0.011255] DMI not present or invalid.785client # [ 0.180532] NET: Registered PF_INET protocol family786worker2 # [ 0.024850] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1787worker1 # [ 0.014406] NET: Registered PF_NETLINK/PF_ROUTE protocol family788worker2 # [ 0.025007] printk: console [ttyAMA0] enabled789client # [ 0.180785] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)790worker1 # [ 0.016873] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations791worker2 # [ 0.144599] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages792worker1 # [ 0.017059] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations793worker2 # [ 0.144618] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page794worker1 # [ 0.017251] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations795worker2 # [ 0.144623] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages796worker1 # [ 0.017272] audit: initializing netlink subsys (disabled)797worker2 # [ 0.144627] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page798worker1 # [ 0.017843] thermal_sys: Registered thermal governor 'fair_share'799worker1 # [ 0.017845] thermal_sys: Registered thermal governor 'bang_bang'800worker2 # [ 0.144631] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages801worker1 # [ 0.017848] thermal_sys: Registered thermal governor 'step_wise'802worker2 # [ 0.144635] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page803worker1 # [ 0.017851] thermal_sys: Registered thermal governor 'user_space'804worker2 # [ 0.144640] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages805worker1 # [ 0.017856] thermal_sys: Registered thermal governor 'power_allocator'806worker2 # [ 0.144644] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page807worker1 # [ 0.017879] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1808worker1 # [ 0.017889] cpuidle: using governor ladder809worker1 # [ 0.017894] cpuidle: using governor menu810worker2 # [ 0.152218] fbcon: Taking over console811worker1 # [ 0.018076] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.812worker2 # [ 0.152232] ACPI: Interpreter disabled.813worker1 # [ 0.018091] ASID allocator initialised with 65536 entries814worker2 # [ 0.154091] iommu: Default domain type: Translated815worker1 # [ 0.019253] Serial: AMBA PL011 UART driver816worker2 # [ 0.154101] iommu: DMA domain TLB invalidation policy: strict mode817worker1 # [ 0.024461] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1818worker2 # [ 0.155789] SCSI subsystem initialized819worker1 # [ 0.024581] printk: console [ttyAMA0] enabled820worker1 # [ 0.144895] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages821worker1 # [ 0.144915] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page822worker1 # [ 0.144921] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages823worker1 # [ 0.144925] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page824worker1 # [ 0.144929] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages825worker1 # [ 0.144934] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page826worker1 # [ 0.144938] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages827worker1 # [ 0.144942] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page828lb # [ 0.211702] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)829worker2 # [ 0.163625] usbcore: registered new interface driver usbfs830lb # [ 0.211741] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)831worker2 # [ 0.163659] usbcore: registered new interface driver hub832worker1 # [ 0.152234] fbcon: Taking over console833worker1 # [ 0.152248] ACPI: Interpreter disabled.834worker2 # [ 0.163682] usbcore: registered new device driver usb835lb # [ 0.211765] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)836worker2 # [ 0.163959] pps_core: LinuxPPS API ver. 1 registered837lb # [ 0.211840] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)838worker2 # [ 0.163965] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>839lb # [ 0.211982] TCP: Hash tables configured (established 16384 bind 16384)840worker2 # [ 0.163976] PTP clock support registered841worker2 # [ 0.164038] EDAC MC: Ver: 3.0.0842lb # [ 0.212056] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)843worker2 # [ 0.168718] scmi_core: SCMI protocol bus registered844lb # [ 0.212102] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)845worker2 # [ 0.169698] FPGA manager framework846lb # [ 0.212136] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)847worker2 # [ 0.170664] vgaarb: loaded848lb # [ 0.212244] NET: Registered PF_UNIX/PF_LOCAL protocol family849lb # [ 0.212307] NET: Registered PF_XDP protocol family850lb # [ 0.212324] PCI: CLS 0 bytes, default 64851worker2 # [ 0.171315] clocksource: Switched to clocksource arch_sys_counter852worker1 # [ 0.160190] iommu: Default domain type: Translated853lb # [ 0.212561] Trying to unpack rootfs image as initramfs...854worker2 # [ 0.171915] VFS: Disk quotas dquot_6.6.0855worker1 # [ 0.160200] iommu: DMA domain TLB invalidation policy: strict mode856worker2 # [ 0.171940] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)857worker1 # [ 0.161917] SCSI subsystem initialized858worker2 # [ 0.174235] netfs: FS-Cache loaded859worker2 # [ 0.174352] pnp: PnP ACPI: disabled860worker1 # [ 0.162594] usbcore: registered new interface driver usbfs861client # [ 0.210015] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)862lb # [ 0.228918] kvm [1]: HYP mode not available863worker1 # [ 0.162625] usbcore: registered new interface driver hub864client # [ 0.210054] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)865worker1 # [ 0.162648] usbcore: registered new device driver usb866worker1 # [ 0.162942] pps_core: LinuxPPS API ver. 1 registered867client # [ 0.210082] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)868client # [ 0.210151] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)869worker1 # [ 0.162948] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>870worker1 # [ 0.162958] PTP clock support registered871client # [ 0.210294] TCP: Hash tables configured (established 16384 bind 16384)872worker1 # [ 0.163008] EDAC MC: Ver: 3.0.0873client # [ 0.210374] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)874worker1 # [ 0.167598] scmi_core: SCMI protocol bus registered875client # [ 0.210409] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)876worker1 # [ 0.168570] FPGA manager framework877worker1 # [ 0.169474] vgaarb: loaded878client # [ 0.210446] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)879client # [ 0.210533] NET: Registered PF_UNIX/PF_LOCAL protocol family880worker1 # [ 0.170093] clocksource: Switched to clocksource arch_sys_counter881client # [ 0.210561] NET: Registered PF_XDP protocol family882client # [ 0.210578] PCI: CLS 0 bytes, default 64883worker2 # [ 0.180435] NET: Registered PF_INET protocol family884client # [ 0.210801] Trying to unpack rootfs image as initramfs...885worker1 # [ 0.172167] VFS: Disk quotas dquot_6.6.0886worker2 # [ 0.180715] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)887worker1 # [ 0.172203] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)888worker1 # [ 0.175958] netfs: FS-Cache loaded889worker1 # [ 0.176093] pnp: PnP ACPI: disabled890client # [ 0.225178] kvm [1]: HYP mode not available891worker1 # [ 0.180039] NET: Registered PF_INET protocol family892worker1 # [ 0.180304] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)893worker2 # [ 0.210909] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)894worker2 # [ 0.210965] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)895worker2 # [ 0.210995] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)896worker2 # [ 0.211068] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)897worker2 # [ 0.211211] TCP: Hash tables configured (established 16384 bind 16384)898worker2 # [ 0.211300] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)899worker1 # [ 0.210074] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)900worker2 # [ 0.216911] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)901worker2 # [ 0.216955] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)902worker2 # [ 0.217056] NET: Registered PF_UNIX/PF_LOCAL protocol family903worker2 # [ 0.217080] NET: Registered PF_XDP protocol family904worker2 # [ 0.217099] PCI: CLS 0 bytes, default 64905worker2 # [ 0.217347] Trying to unpack rootfs image as initramfs...906worker1 # [ 0.211293] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)907worker2 # [ 0.227447] kvm [1]: HYP mode not available908worker1 # [ 0.211326] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909worker1 # [ 0.211399] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910worker1 # [ 0.211542] TCP: Hash tables configured (established 16384 bind 16384)911worker1 # [ 0.211625] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912worker1 # [ 0.211662] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913worker1 # [ 0.211697] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914worker1 # [ 0.211803] NET: Registered PF_UNIX/PF_LOCAL protocol family915worker1 # [ 0.211830] NET: Registered PF_XDP protocol family916worker1 # [ 0.211849] PCI: CLS 0 bytes, default 64917worker1 # [ 0.212083] Trying to unpack rootfs image as initramfs...918worker1 # [ 0.226671] kvm [1]: HYP mode not available919lb # [ 0.313412] Initialise system trusted keyrings920lb # [ 0.319179] workingset: timestamp_bits=42 max_order=19 bucket_order=0921lb # [ 0.320620] squashfs: version 4.0 (2009/01/31) Phillip Lougher922lb # [ 0.321393] 9p: Installing v9fs 9p2000 file system support923client # [ 0.316060] Initialise system trusted keyrings924client # [ 0.316791] workingset: timestamp_bits=42 max_order=19 bucket_order=0925client # [ 0.318039] squashfs: version 4.0 (2009/01/31) Phillip Lougher926client # [ 0.318802] 9p: Installing v9fs 9p2000 file system support927lb # [ 0.342033] Key type asymmetric registered928lb # [ 0.342057] Asymmetric key parser 'x509' registered929lb # [ 0.342144] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)930lb # [ 0.351178] io scheduler mq-deadline registered931lb # [ 0.351198] io scheduler kyber registered932client # [ 0.339436] Key type asymmetric registered933client # [ 0.339460] Asymmetric key parser 'x509' registered934lb # [ 0.356294] pl061_gpio 9030000.pl061: PL061 GPIO chip registered935lb # [ 0.357591] ledtrig-cpu: registered to indicate activity on CPUs936lb # [ 0.357979] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:937lb # [ 0.357996] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000938lb # [ 0.358008] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000939client # [ 0.347569] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)940lb # [ 0.358017] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000941client # [ 0.348665] io scheduler mq-deadline registered942client # [ 0.348677] io scheduler kyber registered943lb # [ 0.358036] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits944lb # [ 0.358057] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]945lb # [ 0.358132] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00946lb # [ 0.358142] pci_bus 0000:00: root bus resource [bus 00-ff]947lb # [ 0.358148] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]948lb # [ 0.358153] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]949lb # [ 0.358158] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]950lb # [ 0.358213] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint951lb # [ 0.358641] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint952lb # [ 0.358830] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]953lb # [ 0.358846] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]954lb # [ 0.358875] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]955lb # [ 0.358891] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]956client # [ 0.359704] pl061_gpio 9030000.pl061: PL061 GPIO chip registered957worker2 # [ 0.321931] Initialise system trusted keyrings958client # [ 0.361099] ledtrig-cpu: registered to indicate activity on CPUs959worker2 # [ 0.322701] workingset: timestamp_bits=42 max_order=19 bucket_order=0960client # [ 0.361526] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:961worker2 # [ 0.324004] squashfs: version 4.0 (2009/01/31) Phillip Lougher962client # [ 0.361544] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000963client # [ 0.361558] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000964client # [ 0.361566] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000965worker2 # [ 0.324913] 9p: Installing v9fs 9p2000 file system support966lb # [ 0.379422] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint967client # [ 0.361587] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits968lb # [ 0.379625] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]969client # [ 0.361614] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]970lb # [ 0.379642] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]971client # [ 0.361697] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00972lb # [ 0.379672] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973client # [ 0.361707] pci_bus 0000:00: root bus resource [bus 00-ff]974lb # [ 0.380155] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint975client # [ 0.361713] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]976worker1 # [ 0.319404] Initialise system trusted keyrings977lb # [ 0.380340] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]978client # [ 0.361718] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]979lb # [ 0.380355] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]980worker1 # [ 0.320133] workingset: timestamp_bits=42 max_order=19 bucket_order=0981client # [ 0.361723] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]982lb # [ 0.380385] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983worker1 # [ 0.321430] squashfs: version 4.0 (2009/01/31) Phillip Lougher984client # [ 0.361777] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint985lb # [ 0.380831] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint986lb # [ 0.381015] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]987client # [ 0.362208] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint988lb # [ 0.381031] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]989client # [ 0.362390] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]990client # [ 0.362405] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]991lb # [ 0.381060] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]992lb # [ 0.381079] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]993client # [ 0.362435] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994client # [ 0.362451] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]995lb # [ 0.381526] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint996worker1 # [ 0.322209] 9p: Installing v9fs 9p2000 file system support997lb # [ 0.381713] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]998client # [ 0.362894] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint999client # [ 0.363077] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1000lb # [ 0.381742] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1001client # [ 0.363092] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1002lb # [ 0.382187] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1003client # [ 0.363121] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1004lb # [ 0.382375] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1005lb # [ 0.382404] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006client # [ 0.382598] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1007lb # [ 0.382845] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1008client # [ 0.382781] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1009lb # [ 0.383029] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1010client # [ 0.382798] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1011client # [ 0.382827] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1012client # [ 0.383285] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1013client # [ 0.383467] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1014client # [ 0.383482] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1015client # [ 0.383511] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1016worker2 # [ 0.348931] Key type asymmetric registered1017worker2 # [ 0.348956] Asymmetric key parser 'x509' registered1018worker2 # [ 0.349035] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1019worker2 # [ 0.351179] io scheduler mq-deadline registered1020worker2 # [ 0.351189] io scheduler kyber registered1021client # [ 0.389762] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1022client # [ 0.390221] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1023client # [ 0.390407] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1024client # [ 0.390438] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025client # [ 0.390883] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1026client # [ 0.391068] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1027lb # [ 0.407372] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1028client # [ 0.391097] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1029lb # [ 0.407583] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1030client # [ 0.391518] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1031lb # [ 0.407614] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1032client # [ 0.391708] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1033lb # [ 0.408093] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1034worker1 # [ 0.350216] Key type asymmetric registered1035client # [ 0.391953] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1036lb # [ 0.408288] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1037worker1 # [ 0.350242] Asymmetric key parser 'x509' registered1038client # [ 0.392136] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1039lb # [ 0.408318] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1040worker1 # [ 0.350321] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1041client # [ 0.392167] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042worker1 # [ 0.352357] io scheduler mq-deadline registered1043lb # [ 0.408769] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1044worker1 # [ 0.352368] io scheduler kyber registered1045client # [ 0.392613] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1046lb # [ 0.408956] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1047client # [ 0.392797] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1048lb # [ 0.408986] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049client # [ 0.392827] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1050lb # [ 0.409442] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1051client # [ 0.393287] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1052lb # [ 0.409723] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1053client # [ 0.393476] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1054lb # [ 0.409740] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1055client # [ 0.393505] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1056lb # [ 0.409769] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057client # [ 0.393960] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1058lb # [ 0.410224] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1059client # [ 0.394254] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1060lb # [ 0.410408] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1061client # [ 0.394270] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1062lb # [ 0.410423] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1063client # [ 0.394301] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064lb # [ 0.410453] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065client # [ 0.394780] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1066lb # [ 0.411032] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1067client # [ 0.394963] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1068lb # [ 0.411044] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1069client # [ 0.394978] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1070worker1 # [ 0.362237] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1071lb # [ 0.411049] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1072worker2 # [ 0.367489] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1073client # [ 0.395007] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074worker2 # [ 0.368143] ledtrig-cpu: registered to indicate activity on CPUs1075lb # [ 0.411095] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1076worker2 # [ 0.368575] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1077lb # [ 0.411159] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1078worker2 # [ 0.368593] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001079worker1 # [ 0.363616] ledtrig-cpu: registered to indicate activity on CPUs1080lb # [ 0.411207] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1081worker1 # [ 0.364023] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1082worker2 # [ 0.368606] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001083lb # [ 0.411254] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1084worker1 # [ 0.364040] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001085lb # [ 0.411301] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1086worker2 # [ 0.368614] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001087lb # [ 0.411350] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1088worker2 # [ 0.368660] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1089worker1 # [ 0.364052] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001090lb # [ 0.411399] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1091worker1 # [ 0.364061] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001092worker2 # [ 0.368687] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1093lb # [ 0.411446] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1094worker2 # [ 0.368763] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001095worker1 # [ 0.364083] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1096worker2 # [ 0.368773] pci_bus 0000:00: root bus resource [bus 00-ff]1097lb # [ 0.411492] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1098worker1 # [ 0.364107] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1099worker2 # [ 0.368779] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1100lb # [ 0.411566] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1101worker1 # [ 0.364196] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001102worker2 # [ 0.368784] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1103worker1 # [ 0.364206] pci_bus 0000:00: root bus resource [bus 00-ff]1104lb # [ 0.411612] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1105worker2 # [ 0.368789] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1106worker1 # [ 0.364212] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1107lb # [ 0.411633] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1108client # [ 0.423639] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1109worker2 # [ 0.368857] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1110lb # [ 0.411655] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1111worker1 # [ 0.364218] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1112client # [ 0.423661] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1113worker2 # [ 0.369305] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1114worker1 # [ 0.364222] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1115lb # [ 0.411676] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1116worker2 # [ 0.369492] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1117client # [ 0.423668] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1118lb # [ 0.411698] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1119worker1 # [ 0.364276] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1120worker2 # [ 0.369508] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1121client # [ 0.423720] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1122lb # [ 0.411720] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1123worker1 # [ 0.364726] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1124worker2 # [ 0.369538] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1125lb # [ 0.411741] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1126client # [ 0.423768] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1127worker1 # [ 0.364915] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1128worker2 # [ 0.369554] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1129worker1 # [ 0.364933] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1130lb # [ 0.411767] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1131client # [ 0.423815] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1132worker2 # [ 0.370007] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1133lb # [ 0.411789] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1134worker1 # [ 0.364964] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135worker2 # [ 0.370191] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1136client # [ 0.423863] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1137lb # [ 0.411811] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1138worker1 # [ 0.364980] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1139worker2 # [ 0.370206] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1140client # [ 0.423910] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1141lb # [ 0.411833] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1142worker1 # [ 0.365439] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1143worker2 # [ 0.370236] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144client # [ 0.423957] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1145worker1 # [ 0.365625] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1146lb # [ 0.411855] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1147worker2 # [ 0.370691] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1148worker1 # [ 0.365641] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1149client # [ 0.424004] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1150lb # [ 0.411876] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1151worker2 # [ 0.370874] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1152lb # [ 0.411897] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1153worker1 # [ 0.365672] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1154client # [ 0.424050] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1155worker2 # [ 0.370889] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1156lb # [ 0.411918] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1157worker2 # [ 0.370919] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158client # [ 0.424098] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1159lb # [ 0.411939] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1160worker2 # [ 0.371397] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1161client # [ 0.424213] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1162lb # [ 0.411961] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1163worker2 # [ 0.371582] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1164worker1 # [ 0.384485] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1165lb # [ 0.411982] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1166client # [ 0.424266] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1167worker1 # [ 0.384676] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1168worker2 # [ 0.371598] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1169lb # [ 0.412007] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1170client # [ 0.424288] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1171worker1 # [ 0.384693] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1172worker2 # [ 0.371627] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1173lb # [ 0.412017] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1174client # [ 0.424309] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1175worker1 # [ 0.384723] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176worker2 # [ 0.371645] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1177lb # [ 0.412021] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1178client # [ 0.424330] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1179worker1 # [ 0.385193] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1180worker2 # [ 0.372100] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1181lb # [ 0.412845] pci 0000:00:07.0: enabling device (0000 -> 0002)1182client # [ 0.424351] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1183worker1 # [ 0.385379] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1184worker2 # [ 0.372287] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1185client # [ 0.424372] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1186worker1 # [ 0.385396] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1187worker2 # [ 0.372316] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1188client # [ 0.424393] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1189worker1 # [ 0.385426] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190worker2 # [ 0.372787] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1191worker1 # [ 0.385446] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1192client # [ 0.424419] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1193worker2 # [ 0.372974] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1194client # [ 0.424441] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1195worker1 # [ 0.385905] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1196worker2 # [ 0.373004] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1197client # [ 0.424463] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1198worker2 # [ 0.373430] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1199client # [ 0.424485] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1200worker2 # [ 0.373613] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1201client # [ 0.424507] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1202client # [ 0.424528] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1203worker2 # [ 0.373872] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1204worker2 # [ 0.374058] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1205client # [ 0.424549] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1206client # [ 0.424570] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1207worker2 # [ 0.374088] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1208client # [ 0.424591] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1209worker2 # [ 0.374545] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1210client # [ 0.424611] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1211worker2 # [ 0.374732] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1212client # [ 0.424632] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1213worker2 # [ 0.374762] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1214client # [ 0.424660] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1215worker2 # [ 0.375222] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216client # [ 0.424669] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1217client # [ 0.424673] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1218worker1 # [ 0.398126] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1219client # [ 0.425490] pci 0000:00:07.0: enabling device (0000 -> 0002)1220worker1 # [ 0.398195] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1221worker2 # [ 0.414025] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1222worker1 # [ 0.398691] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1223worker2 # [ 0.414059] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1224worker1 # [ 0.398889] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1225worker2 # [ 0.414530] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1226worker1 # [ 0.398920] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1227worker2 # [ 0.414831] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1228worker2 # [ 0.414848] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1229worker1 # [ 0.399358] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1230worker1 # [ 0.399543] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1231worker2 # [ 0.414877] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1232worker1 # [ 0.399795] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1233worker1 # [ 0.399985] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1234worker1 # [ 0.400015] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1235worker1 # [ 0.400481] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1236worker1 # [ 0.400670] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1237worker1 # [ 0.400701] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1238worker1 # [ 0.401155] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1239worker1 # [ 0.401347] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1240worker1 # [ 0.401378] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1241lb # [ 0.481631] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1242worker1 # [ 0.401836] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1243lb # [ 0.484630] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1244worker2 # [ 0.427442] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1245lb # [ 0.486674] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1246worker2 # [ 0.427653] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1247worker2 # [ 0.427671] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1248worker2 # [ 0.427701] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1249worker2 # [ 0.428325] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1250worker2 # [ 0.428337] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1251worker2 # [ 0.428343] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1252worker1 # [ 0.422239] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1253worker2 # [ 0.428389] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1254worker1 # [ 0.422269] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1255worker2 # [ 0.428453] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1256worker1 # [ 0.422300] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1257worker2 # [ 0.428500] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1258worker1 # [ 0.422792] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1259worker1 # [ 0.422983] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1260worker2 # [ 0.428548] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1261worker1 # [ 0.423000] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1262worker2 # [ 0.428595] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1263worker1 # [ 0.423031] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1264client # [ 0.481865] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1265worker2 # [ 0.428641] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1266lb # [ 0.496545] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1267worker1 # [ 0.423641] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1268lb # [ 0.498531] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1269worker2 # [ 0.428687] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1270worker1 # [ 0.423653] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1271worker2 # [ 0.428732] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1272worker1 # [ 0.423659] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1273worker2 # [ 0.428778] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1274worker1 # [ 0.423707] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1275client # [ 0.484794] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1276worker2 # [ 0.428897] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1277worker1 # [ 0.423756] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1278client # [ 0.486940] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1279worker2 # [ 0.428944] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1280worker1 # [ 0.423805] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1281worker2 # [ 0.428965] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1282worker1 # [ 0.423856] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1283worker2 # [ 0.428985] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1284worker1 # [ 0.423905] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1285worker2 # [ 0.429006] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1286worker1 # [ 0.423954] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1287worker2 # [ 0.429027] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1288worker2 # [ 0.429047] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1289worker1 # [ 0.424003] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1290worker2 # [ 0.429068] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1291worker1 # [ 0.424051] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1292worker2 # [ 0.429093] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1293worker1 # [ 0.424099] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1294worker2 # [ 0.429114] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1295worker1 # [ 0.424188] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1296lb # [ 0.509921] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1297worker2 # [ 0.429135] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1298worker1 # [ 0.424301] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1299worker2 # [ 0.429157] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1300worker1 # [ 0.424325] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1301worker2 # [ 0.429179] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1302worker1 # [ 0.424348] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1303worker2 # [ 0.429199] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1304worker1 # [ 0.424370] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1305worker2 # [ 0.429220] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1306worker1 # [ 0.424393] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1307lb # [ 0.512467] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1308worker2 # [ 0.429240] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1309worker2 # [ 0.429260] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1310lb # [ 0.514231] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1311worker1 # [ 0.424425] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1312worker2 # [ 0.429281] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1313worker1 # [ 0.424448] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1314worker2 # [ 0.429301] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1315worker1 # [ 0.424474] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1316worker2 # [ 0.429327] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1317worker1 # [ 0.424497] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1318client # [ 0.500973] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1319worker2 # [ 0.429336] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1320worker1 # [ 0.424520] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1321client # [ 0.503055] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1322worker2 # [ 0.429341] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1323worker1 # [ 0.424543] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1324worker2 # [ 0.430143] pci 0000:00:07.0: enabling device (0000 -> 0002)1325worker1 # [ 0.424567] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1326worker1 # [ 0.424589] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1327worker1 # [ 0.424611] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1328client # [ 0.506764] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1329worker1 # [ 0.424633] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1330worker1 # [ 0.424655] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1331worker1 # [ 0.424677] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1332worker1 # [ 0.424698] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1333worker1 # [ 0.424726] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1334worker1 # [ 0.424737] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1335lb # [ 0.524509] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1336worker1 # [ 0.424741] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1337client # [ 0.509544] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1338lb # [ 0.526406] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1339worker1 # [ 0.425568] pci 0000:00:07.0: enabling device (0000 -> 0002)1340client # [ 0.511434] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1341lb # [ 0.530119] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1342client # [ 0.520804] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1343client # [ 0.522533] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1344worker2 # [ 0.483836] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1345worker2 # [ 0.486104] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1346lb # [ 0.548590] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1347client # [ 0.533033] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1348lb # [ 0.551040] msm_serial: driver initialized1349worker2 # [ 0.497178] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1350worker2 # [ 0.499115] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1351client # [ 0.542034] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1352worker1 # [ 0.488611] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1353lb # [ 0.555261] SuperH (H)SCI(F) driver initialized1354lb # [ 0.555328] STM32 USART driver initialized1355worker1 # [ 0.491588] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1356worker1 # [ 0.493620] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1357client # [ 0.548768] msm_serial: driver initialized1358client # [ 0.548911] SuperH (H)SCI(F) driver initialized1359worker2 # [ 0.510255] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1360client # [ 0.548964] STM32 USART driver initialized1361worker1 # [ 0.503489] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1362worker1 # [ 0.505496] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1363worker1 # [ 0.509063] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1364worker2 # [ 0.520214] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1365worker2 # [ 0.524322] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1366worker2 # [ 0.526413] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1367worker1 # [ 0.515834] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1368lb # [ 0.587371] loop: module loaded1369lb # [ 0.587560] virtio_blk virtio2: 1/0/0 default/read/poll queues1370lb # [ 0.588545] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1371worker1 # [ 0.522678] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1372worker2 # [ 0.537613] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1373worker1 # [ 0.524815] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1374client # [ 0.580312] loop: module loaded1375client # [ 0.580494] virtio_blk virtio2: 1/0/0 default/read/poll queues1376client # [ 0.581350] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1377worker2 # [ 0.540361] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1378worker2 # [ 0.542805] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1379lb # [ 0.595729] megasas: 07.734.00.00-rc11380lb # [ 0.596484] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1381lb # [ 0.598515] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001382worker1 # [ 0.534644] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1383lb # [ 0.598539] Intel/Sharp Extended Query Table at 0x00311384worker1 # [ 0.536969] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1385lb # [ 0.600312] Using buffer write method1386lb # [ 0.600368] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1387lb # [ 0.602121] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001388lb # [ 0.602160] Intel/Sharp Extended Query Table at 0x00311389client # [ 0.592156] megasas: 07.734.00.00-rc11390client # [ 0.592887] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1391client # [ 0.594861] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001392worker2 # [ 0.560488] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1393client # [ 0.594886] Intel/Sharp Extended Query Table at 0x00311394client # [ 0.596641] Using buffer write method1395worker2 # [ 0.563075] msm_serial: driver initialized1396worker2 # [ 0.563213] SuperH (H)SCI(F) driver initialized1397worker2 # [ 0.563266] STM32 USART driver initialized1398client # [ 0.596718] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1399lb # [ 0.620001] Using buffer write method1400lb # [ 0.620037] Concatenating MTD devices:1401lb # [ 0.620041] (0): "0.flash"1402client # [ 0.598610] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001403lb # [ 0.620045] (1): "0.flash"1404lb # [ 0.620066] into device "0.flash"1405client # [ 0.598632] Intel/Sharp Extended Query Table at 0x00311406worker1 # [ 0.555277] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1407worker1 # [ 0.557962] msm_serial: driver initialized1408worker1 # [ 0.558658] SuperH (H)SCI(F) driver initialized1409worker1 # [ 0.558719] STM32 USART driver initialized1410client # [ 0.616341] Using buffer write method1411client # [ 0.616377] Concatenating MTD devices:1412client # [ 0.616382] (0): "0.flash"1413client # [ 0.616386] (1): "0.flash"1414client # [ 0.616394] into device "0.flash"1415worker2 # [ 0.597245] loop: module loaded1416worker2 # [ 0.597421] virtio_blk virtio2: 1/0/0 default/read/poll queues1417worker2 # [ 0.598208] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1418worker1 # [ 0.592514] loop: module loaded1419worker1 # [ 0.592701] virtio_blk virtio2: 1/0/0 default/read/poll queues1420worker2 # [ 0.603928] megasas: 07.734.00.00-rc11421worker1 # [ 0.593458] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1422worker2 # [ 0.604706] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1423worker2 # [ 0.606936] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001424worker2 # [ 0.606962] Intel/Sharp Extended Query Table at 0x00311425worker1 # [ 0.598681] megasas: 07.734.00.00-rc11426worker1 # [ 0.599345] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1427worker2 # [ 0.616367] Using buffer write method1428worker2 # [ 0.616453] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1429worker1 # [ 0.610136] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001430worker1 # [ 0.610161] Intel/Sharp Extended Query Table at 0x00311431worker1 # [ 0.611863] Using buffer write method1432worker1 # [ 0.611932] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1433worker2 # [ 0.627358] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001434worker2 # [ 0.627383] Intel/Sharp Extended Query Table at 0x00311435worker2 # [ 0.628929] Using buffer write method1436worker1 # [ 0.617980] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001437worker2 # [ 0.628959] Concatenating MTD devices:1438worker1 # [ 0.618039] Intel/Sharp Extended Query Table at 0x00311439worker2 # [ 0.628963] (0): "0.flash"1440worker2 # [ 0.628967] (1): "0.flash"1441worker2 # [ 0.628977] into device "0.flash"1442worker1 # [ 0.627917] Using buffer write method1443worker1 # [ 0.627952] Concatenating MTD devices:1444worker1 # [ 0.627956] (0): "0.flash"1445worker1 # [ 0.627960] (1): "0.flash"1446worker1 # [ 0.627969] into device "0.flash"1447lb # [ 0.875566] Freeing initrd memory: 26948K1448client # [ 0.862859] Freeing initrd memory: 26972K1449lb # [ 0.881644] tun: Universal TUN/TAP device driver, 1.61450client # [ 0.868898] tun: Universal TUN/TAP device driver, 1.61451lb # [ 0.885489] thunder_xcv, ver 1.01452lb # [ 0.885533] thunder_bgx, ver 1.01453lb # [ 0.885556] nicpf, ver 1.01454lb # [ 0.886123] e1000: Intel(R) PRO/1000 Network Driver1455lb # [ 0.886130] e1000: Copyright (c) 1999-2006 Intel Corporation.1456lb # [ 0.886154] e1000e: Intel(R) PRO/1000 Network Driver1457lb # [ 0.886161] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1458lb # [ 0.886190] igb: Intel(R) Gigabit Ethernet Network Driver1459client # [ 0.872709] thunder_xcv, ver 1.01460client # [ 0.872751] thunder_bgx, ver 1.01461lb # [ 0.886195] igb: Copyright (c) 2007-2014 Intel Corporation.1462client # [ 0.872772] nicpf, ver 1.01463lb # [ 0.886217] igbvf: Intel(R) Gigabit Virtual Function Network Driver1464client # [ 0.873334] e1000: Intel(R) PRO/1000 Network Driver1465lb # [ 0.886223] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1466client # [ 0.873341] e1000: Copyright (c) 1999-2006 Intel Corporation.1467lb # [ 0.886359] sky2: driver version 1.301468client # [ 0.873364] e1000e: Intel(R) PRO/1000 Network Driver1469client # [ 0.873373] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1470client # [ 0.873400] igb: Intel(R) Gigabit Ethernet Network Driver1471client # [ 0.873406] igb: Copyright (c) 2007-2014 Intel Corporation.1472client # [ 0.873427] igbvf: Intel(R) Gigabit Virtual Function Network Driver1473lb # [ 0.895207] usbcore: registered new interface driver usb-storage1474client # [ 0.873433] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1475lb # [ 0.895303] usbcore: registered new interface driver usbserial_generic1476client # [ 0.873562] sky2: driver version 1.301477lb # [ 0.895319] usbserial: USB Serial support registered for generic1478client # [ 0.875084] usbcore: registered new interface driver usb-storage1479lb # [ 0.895951] hv_vmbus: registering driver hyperv_keyboard1480client # [ 0.875198] usbcore: registered new interface driver usbserial_generic1481client # [ 0.875212] usbserial: USB Serial support registered for generic1482lb # [ 0.900220] ehci-pci 0000:00:07.0: EHCI Host Controller1483client # [ 0.876221] ehci-pci 0000:00:07.0: EHCI Host Controller1484lb # [ 0.900250] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11485client # [ 0.876246] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11486lb # [ 0.900408] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001487client # [ 0.876526] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001488lb # [ 0.902916] rtc-pl031 9010000.pl031: registered as rtc01489client # [ 0.888063] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001490lb # [ 0.902945] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T13:45:38 UTC (1790084738)1491client # [ 0.889402] hv_vmbus: registering driver hyperv_keyboard1492client # [ 0.890105] hub 1-0:1.0: USB hub found1493lb # [ 0.905072] i2c_dev: i2c /dev entries driver1494client # [ 0.890125] hub 1-0:1.0: 6 ports detected1495client # [ 0.892307] rtc-pl031 9010000.pl031: registered as rtc01496client # [ 0.892334] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T13:45:38 UTC (1790084738)1497client # [ 0.892637] i2c_dev: i2c /dev entries driver1498lb # [ 0.908804] sdhci: Secure Digital Host Controller Interface driver1499lb # [ 0.908814] sdhci: Copyright(c) Pierre Ossman1500lb # [ 0.909082] Synopsys Designware Multimedia Card Interface Driver1501lb # [ 0.909455] sdhci-pltfm: SDHCI platform and OF driver helper1502lb # [ 0.910993] hid: raw HID events driver (C) Jiri Kosina1503client # [ 0.897949] sdhci: Secure Digital Host Controller Interface driver1504client # [ 0.897960] sdhci: Copyright(c) Pierre Ossman1505lb # [ 0.911162] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001506lb # [ 0.911467] hub 1-0:1.0: USB hub found1507client # [ 0.898222] Synopsys Designware Multimedia Card Interface Driver1508lb # [ 0.911488] hub 1-0:1.0: 6 ports detected1509client # [ 0.898594] sdhci-pltfm: SDHCI platform and OF driver helper1510lb # [ 0.916962] usbcore: registered new interface driver usbhid1511client # [ 0.902782] hid: raw HID events driver (C) Jiri Kosina1512lb # [ 0.916971] usbhid: USB HID core driver1513client # [ 0.903024] usbcore: registered new interface driver usbhid1514client # [ 0.903030] usbhid: USB HID core driver1515lb # [ 0.919306] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1516lb # [ 0.920885] drop_monitor: Initializing network drop monitor service1517client # [ 0.905896] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1518lb # [ 0.921027] NET: Registered PF_INET6 protocol family1519client # [ 0.907374] drop_monitor: Initializing network drop monitor service1520lb # [ 0.924066] Segment Routing with IPv61521client # [ 0.907519] NET: Registered PF_INET6 protocol family1522lb # [ 0.924083] In-situ OAM (IOAM) with IPv61523lb # [ 0.924134] NET: Registered PF_PACKET protocol family1524client # [ 0.910546] Segment Routing with IPv61525client # [ 0.910577] In-situ OAM (IOAM) with IPv61526client # [ 0.910605] NET: Registered PF_PACKET protocol family1527lb # [ 0.925828] 9pnet: Installing 9P2000 support1528lb # [ 0.925874] Key type dns_resolver registered1529client # [ 0.912238] 9pnet: Installing 9P2000 support1530client # [ 0.912285] Key type dns_resolver registered1531client # [ 0.919059] registered taskstats version 11532client # [ 0.919204] Loading compiled-in X.509 certificates1533lb # [ 0.932532] registered taskstats version 11534lb # [ 0.932667] Loading compiled-in X.509 certificates1535lb # [ 0.941620] Demotion targets for Node 0: null1536lb # [ 0.941738] Key type .fscrypt registered1537lb # [ 0.941744] Key type fscrypt-provisioning registered1538client # [ 0.928146] Demotion targets for Node 0: null1539worker1 # [ 0.874266] Freeing initrd memory: 26976K1540lb # [ 0.941842] ima: No TPM chip found, activating TPM-bypass!1541client # [ 0.928248] Key type .fscrypt registered1542worker2 # [ 0.886718] Freeing initrd memory: 26944K1543lb # [ 0.941863] ima: Allocated hash algorithm: sha11544client # [ 0.928255] Key type fscrypt-provisioning registered1545lb # [ 0.941885] ima: No architecture policies found1546client # [ 0.928348] ima: No TPM chip found, activating TPM-bypass!1547client # [ 0.928367] ima: Allocated hash algorithm: sha11548lb # [ 0.945913] input: gpio-keys as /devices/platform/gpio-keys/input/input01549client # [ 0.928387] ima: No architecture policies found1550client # [ 0.932588] input: gpio-keys as /devices/platform/gpio-keys/input/input01551worker1 # [ 0.880320] tun: Universal TUN/TAP device driver, 1.61552worker2 # [ 0.892837] tun: Universal TUN/TAP device driver, 1.61553worker1 # [ 0.884034] thunder_xcv, ver 1.01554worker1 # [ 0.884079] thunder_bgx, ver 1.01555worker1 # [ 0.884100] nicpf, ver 1.01556worker1 # [ 0.884683] e1000: Intel(R) PRO/1000 Network Driver1557worker1 # [ 0.884691] e1000: Copyright (c) 1999-2006 Intel Corporation.1558worker2 # [ 0.896677] thunder_xcv, ver 1.01559worker1 # [ 0.884713] e1000e: Intel(R) PRO/1000 Network Driver1560worker2 # [ 0.896730] thunder_bgx, ver 1.01561worker2 # [ 0.896754] nicpf, ver 1.01562worker1 # [ 0.884723] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1563worker2 # [ 0.897315] e1000: Intel(R) PRO/1000 Network Driver1564worker1 # [ 0.884752] igb: Intel(R) Gigabit Ethernet Network Driver1565worker2 # [ 0.897323] e1000: Copyright (c) 1999-2006 Intel Corporation.1566worker1 # [ 0.884759] igb: Copyright (c) 2007-2014 Intel Corporation.1567worker2 # [ 0.897348] e1000e: Intel(R) PRO/1000 Network Driver1568worker1 # [ 0.884779] igbvf: Intel(R) Gigabit Virtual Function Network Driver1569worker2 # [ 0.897357] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1570worker1 # [ 0.884785] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1571worker2 # [ 0.897388] igb: Intel(R) Gigabit Ethernet Network Driver1572worker1 # [ 0.884919] sky2: driver version 1.301573worker2 # [ 0.897393] igb: Copyright (c) 2007-2014 Intel Corporation.1574worker2 # [ 0.897421] igbvf: Intel(R) Gigabit Virtual Function Network Driver1575worker1 # [ 0.894009] ehci-pci 0000:00:07.0: EHCI Host Controller1576worker2 # [ 0.897426] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1577worker2 # [ 0.897560] sky2: driver version 1.301578worker1 # [ 0.894043] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11579worker2 # [ 0.899179] usbcore: registered new interface driver usb-storage1580worker1 # [ 0.894289] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001581worker2 # [ 0.899228] usbcore: registered new interface driver usbserial_generic1582worker2 # [ 0.899242] usbserial: USB Serial support registered for generic1583lb # [ 0.963152] clk: Disabling unused clocks1584worker2 # [ 0.900121] ehci-pci 0000:00:07.0: EHCI Host Controller1585lb # [ 0.963170] PM: genpd: Disabling unused power domains1586worker1 # [ 0.897106] usbcore: registered new interface driver usb-storage1587client # [ 0.949819] clk: Disabling unused clocks1588worker2 # [ 0.900149] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11589worker1 # [ 0.897198] usbcore: registered new interface driver usbserial_generic1590client # [ 0.949842] PM: genpd: Disabling unused power domains1591worker1 # [ 0.897212] usbserial: USB Serial support registered for generic1592lb # [ 0.967311] Freeing unused kernel memory: 4736K1593worker2 # [ 0.900475] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001594worker1 # [ 0.897807] hv_vmbus: registering driver hyperv_keyboard1595lb # [ 0.967507] Run /init as init process1596worker2 # [ 0.911456] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001597worker2 # [ 0.911763] hub 1-0:1.0: USB hub found1598client # [ 0.953963] Freeing unused kernel memory: 4736K1599worker2 # [ 0.911783] hub 1-0:1.0: 6 ports detected1600client # [ 0.954182] Run /init as init process1601worker1 # [ 0.901581] rtc-pl031 9010000.pl031: registered as rtc01602worker1 # [ 0.901609] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T13:45:38 UTC (1790084738)1603worker2 # [ 0.915066] hv_vmbus: registering driver hyperv_keyboard1604worker1 # [ 0.901905] i2c_dev: i2c /dev entries driver1605worker2 # [ 0.916687] rtc-pl031 9010000.pl031: registered as rtc01606worker2 # [ 0.916717] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T13:45:38 UTC (1790084738)1607worker2 # [ 0.917028] i2c_dev: i2c /dev entries driver1608worker1 # [ 0.907213] sdhci: Secure Digital Host Controller Interface driver1609worker1 # [ 0.907225] sdhci: Copyright(c) Pierre Ossman1610worker1 # [ 0.907493] Synopsys Designware Multimedia Card Interface Driver1611worker2 # [ 0.922499] sdhci: Secure Digital Host Controller Interface driver1612worker2 # [ 0.922512] sdhci: Copyright(c) Pierre Ossman1613worker1 # [ 0.907868] sdhci-pltfm: SDHCI platform and OF driver helper1614worker2 # [ 0.922783] Synopsys Designware Multimedia Card Interface Driver1615worker1 # [ 0.909401] hid: raw HID events driver (C) Jiri Kosina1616worker1 # [ 0.909649] usbcore: registered new interface driver usbhid1617worker2 # [ 0.923158] sdhci-pltfm: SDHCI platform and OF driver helper1618lb # [ 0.981320] systemd[1]: Successfully made /usr/ read-only.1619worker1 # [ 0.909656] usbhid: USB HID core driver1620worker1 # [ 0.914627] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001621worker1 # [ 0.914986] hub 1-0:1.0: USB hub found1622worker1 # [ 0.916166] hub 1-0:1.0: 6 ports detected1623worker2 # [ 0.927530] hid: raw HID events driver (C) Jiri Kosina1624client # [ 0.969747] systemd[1]: Successfully made /usr/ read-only.1625worker2 # [ 0.927775] usbcore: registered new interface driver usbhid1626worker2 # [ 0.927781] usbhid: USB HID core driver1627worker1 # [ 0.917714] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1628worker2 # [ 0.930719] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1629worker1 # [ 0.920337] drop_monitor: Initializing network drop monitor service1630worker1 # [ 0.920639] NET: Registered PF_INET6 protocol family1631worker2 # [ 0.933355] drop_monitor: Initializing network drop monitor service1632worker2 # [ 0.933500] NET: Registered PF_INET6 protocol family1633worker1 # [ 0.922558] Segment Routing with IPv61634worker1 # [ 0.922591] In-situ OAM (IOAM) with IPv61635worker1 # [ 0.922619] NET: Registered PF_PACKET protocol family1636worker2 # [ 0.935656] Segment Routing with IPv61637worker2 # [ 0.935676] In-situ OAM (IOAM) with IPv61638worker2 # [ 0.935703] NET: Registered PF_PACKET protocol family1639worker1 # [ 0.924246] 9pnet: Installing 9P2000 support1640worker1 # [ 0.924294] Key type dns_resolver registered1641worker2 # [ 0.937325] 9pnet: Installing 9P2000 support1642worker2 # [ 0.937371] Key type dns_resolver registered1643worker1 # [ 0.931393] registered taskstats version 11644worker1 # [ 0.931548] Loading compiled-in X.509 certificates1645worker2 # [ 0.943951] registered taskstats version 11646worker2 # [ 0.944099] Loading compiled-in X.509 certificates1647worker1 # [ 0.940254] Demotion targets for Node 0: null1648worker2 # [ 0.952691] Demotion targets for Node 0: null1649worker1 # [ 0.940359] Key type .fscrypt registered1650worker2 # [ 0.952795] Key type .fscrypt registered1651worker1 # [ 0.940365] Key type fscrypt-provisioning registered1652worker2 # [ 0.952802] Key type fscrypt-provisioning registered1653worker1 # [ 0.940473] ima: No TPM chip found, activating TPM-bypass!1654worker2 # [ 0.952896] ima: No TPM chip found, activating TPM-bypass!1655worker1 # [ 0.940493] ima: Allocated hash algorithm: sha11656worker2 # [ 0.952916] ima: Allocated hash algorithm: sha11657worker1 # [ 0.940513] ima: No architecture policies found1658worker2 # [ 0.952938] ima: No architecture policies found1659worker1 # [ 0.944509] input: gpio-keys as /devices/platform/gpio-keys/input/input01660worker2 # [ 0.956966] input: gpio-keys as /devices/platform/gpio-keys/input/input01661worker1 # [ 0.961969] clk: Disabling unused clocks1662worker1 # [ 0.961996] PM: genpd: Disabling unused power domains1663worker2 # [ 0.975820] clk: Disabling unused clocks1664worker2 # [ 0.975848] PM: genpd: Disabling unused power domains1665worker1 # [ 0.966146] Freeing unused kernel memory: 4736K1666worker1 # [ 0.966341] Run /init as init process1667worker2 # [ 0.980022] Freeing unused kernel memory: 4736K1668worker2 # [ 0.980214] Run /init as init process1669worker1 # [ 0.980282] systemd[1]: Successfully made /usr/ read-only.1670worker2 # [ 0.997611] systemd[1]: Successfully made /usr/ read-only.1671client # [ 1.139611] usb 1-1: new high-speed USB device number 2 using ehci-pci1672lb # [ 1.159205] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker2 # [ 1.159419] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker1 # [ 1.162178] usb 1-1: new high-speed USB device number 2 using ehci-pci1675client # [ 1.292238] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676lb # [ 1.313712] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677client # [ 1.304925] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678lb # [ 1.319964] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1679client # [ 1.317271] systemd[1]: Detected virtualization qemu.1680lb # [ 1.320024] systemd[1]: Detected virtualization qemu.1681client # [ 1.319393] systemd[1]: Detected architecture arm64.1682lb # [ 1.320121] systemd[1]: Detected architecture arm64.1683client # [ 1.321468] systemd[1]: Running in initrd.1684lb # [ 1.320144] systemd[1]: Running in initrd.1685client # [ 1.324145] systemd[1]: Initializing machine ID from random generator.1686lb # [ 1.321091] systemd[1]: Initializing machine ID from random generator.1687lb # [ 1.321404] systemd[1]: Hostname set to <lb>.1688client # [ 1.327097] systemd[1]: Hostname set to <client>.1689worker2 # [ 1.309776] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11690worker1 # [ 1.314606] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11691worker2 # [ 1.332237] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692worker2 # [ 1.337668] systemd[1]: Detected virtualization qemu.1693client # [ 1.375968] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01694worker2 # [ 1.338553] systemd[1]: Detected architecture arm64.1695worker2 # [ 1.339373] systemd[1]: Running in initrd.1696worker2 # [ 1.340632] systemd[1]: Initializing machine ID from random generator.1697worker2 # [ 1.341871] systemd[1]: Hostname set to <worker2>.1698worker1 # [ 1.320728] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1699lb # [ 1.399377] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01700worker1 # [ 1.333309] systemd[1]: Detected virtualization qemu.1701worker1 # [ 1.333402] systemd[1]: Detected architecture arm64.1702worker1 # [ 1.333426] systemd[1]: Running in initrd.1703worker1 # [ 1.339884] systemd[1]: Initializing machine ID from random generator.1704worker1 # [ 1.342790] systemd[1]: Hostname set to <worker1>.1705worker2 # [ 1.391647] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker1 # [ 1.398362] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707client # [ 1.499611] usb 1-2: new high-speed USB device number 3 using ehci-pci1708lb # [ 1.519179] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.515398] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker1 # [ 1.518200] usb 1-2: new high-speed USB device number 3 using ehci-pci1711client # [ 1.635066] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712lb # [ 1.650288] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713client # [ 1.660379] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21714lb # [ 1.675909] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21715client # [ 1.666168] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01716lb # [ 1.683286] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01717worker2 # [ 1.651544] systemd[1]: bpf-restrict-fs: LSM BPF program attached1718worker1 # [ 1.650560] systemd[1]: bpf-restrict-fs: LSM BPF program attached1719worker2 # [ 1.679205] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21720worker2 # [ 1.684364] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01721worker1 # [ 1.678999] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21722worker1 # [ 1.684652] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723client # [ 1.750987] systemd[1]: Queued start job for default target Initrd Default Target.1724lb # [ 1.766144] systemd[1]: Queued start job for default target Initrd Default Target.1725client # [ 1.762014] systemd[1]: Created slice Slice /system/modprobe.1726lb # [ 1.777617] systemd[1]: Created slice Slice /system/modprobe.1727client # [ 1.763212] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1728lb # [ 1.778828] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1729client # [ 1.764614] systemd[1]: Expecting device /dev/disk/by-label/nixos...1730lb # [ 1.780277] systemd[1]: Expecting device /dev/disk/by-label/nixos...1731client # [ 1.765688] systemd[1]: Reached target Path Units.1732lb # [ 1.781350] systemd[1]: Reached target Path Units.1733client # [ 1.766506] systemd[1]: Reached target Slice Units.1734lb # [ 1.782169] systemd[1]: Reached target Slice Units.1735client # [ 1.767425] systemd[1]: Reached target Swaps.1736lb # [ 1.783114] systemd[1]: Reached target Swaps.1737client # [ 1.768235] systemd[1]: Reached target Timer Units.1738lb # [ 1.783895] systemd[1]: Reached target Timer Units.1739client # [ 1.769260] systemd[1]: Listening on D-Bus System Message Bus Socket.1740lb # [ 1.784928] systemd[1]: Listening on D-Bus System Message Bus Socket.1741client # [ 1.770492] systemd[1]: Listening on Journal Socket (/dev/log).1742lb # [ 1.786169] systemd[1]: Listening on Journal Socket (/dev/log).1743client # [ 1.771655] systemd[1]: Listening on Journal Sockets.1744client # [ 1.772646] systemd[1]: Listening on udev Control Socket.1745client # [ 1.773676] systemd[1]: Listening on udev Kernel Socket.1746lb # [ 1.787338] systemd[1]: Listening on Journal Sockets.1747client # [ 1.774584] systemd[1]: Reached target Socket Units.1748lb # [ 1.787481] systemd[1]: Listening on udev Control Socket.1749lb # [ 1.787595] systemd[1]: Listening on udev Kernel Socket.1750lb # [ 1.787619] systemd[1]: Reached target Socket Units.1751client # [ 1.777062] systemd[1]: Starting Create List of Static Device Nodes...1752lb # [ 1.792655] systemd[1]: Starting Create List of Static Device Nodes...1753client # [ 1.778242] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1754lb # [ 1.793812] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1755client # [ 1.787708] systemd[1]: Mounting Kernel Configuration File System...1756lb # [ 1.803323] systemd[1]: Mounting Kernel Configuration File System...1757client # [ 1.795828] systemd[1]: Starting Journal Service...1758lb # [ 1.811431] systemd[1]: Starting Journal Service...1759worker2 # [ 1.758669] systemd[1]: Queued start job for default target Initrd Default Target.1760worker2 # [ 1.769505] systemd[1]: Created slice Slice /system/modprobe.1761worker1 # [ 1.758670] systemd[1]: Queued start job for default target Initrd Default Target.1762worker2 # [ 1.770655] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1763worker2 # [ 1.771859] systemd[1]: Expecting device /dev/disk/by-label/nixos...1764worker2 # [ 1.772853] systemd[1]: Reached target Path Units.1765worker2 # [ 1.773555] systemd[1]: Reached target Slice Units.1766worker2 # [ 1.774274] systemd[1]: Reached target Swaps.1767worker2 # [ 1.774930] systemd[1]: Reached target Timer Units.1768worker2 # [ 1.775869] systemd[1]: Listening on D-Bus System Message Bus Socket.1769worker2 # [ 1.776950] systemd[1]: Listening on Journal Socket (/dev/log).1770worker2 # [ 1.777937] systemd[1]: Listening on Journal Sockets.1771worker2 # [ 1.778791] systemd[1]: Listening on udev Control Socket.1772client # [ 1.819760] systemd[1]: Starting Load Kernel Modules...1773lb # [ 1.835616] systemd[1]: Starting Load Kernel Modules...1774worker1 # [ 1.768274] systemd[1]: Created slice Slice /system/modprobe.1775client # [ 1.820718] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1776worker2 # [ 1.779723] systemd[1]: Listening on udev Kernel Socket.1777worker2 # [ 1.779747] systemd[1]: Reached target Socket Units.1778lb # [ 1.836647] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1779worker1 # [ 1.769370] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1780worker2 # [ 1.782618] systemd[1]: Starting Create List of Static Device Nodes...1781worker1 # [ 1.770616] systemd[1]: Expecting device /dev/disk/by-label/nixos...1782worker1 # [ 1.770650] systemd[1]: Reached target Path Units.1783worker2 # [ 1.783680] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1784worker1 # [ 1.770668] systemd[1]: Reached target Slice Units.1785worker1 # [ 1.770687] systemd[1]: Reached target Swaps.1786worker1 # [ 1.770707] systemd[1]: Reached target Timer Units.1787worker1 # [ 1.770875] systemd[1]: Listening on D-Bus System Message Bus Socket.1788worker1 # [ 1.771012] systemd[1]: Listening on Journal Socket (/dev/log).1789worker1 # [ 1.771138] systemd[1]: Listening on Journal Sockets.1790worker1 # [ 1.771255] systemd[1]: Listening on udev Control Socket.1791worker1 # [ 1.771355] systemd[1]: Listening on udev Kernel Socket.1792worker2 # [ 1.791479] systemd[1]: Mounting Kernel Configuration File System...1793worker1 # [ 1.771375] systemd[1]: Reached target Socket Units.1794worker1 # [ 1.781280] systemd[1]: Starting Create List of Static Device Nodes...1795worker1 # [ 1.782336] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1796worker2 # [ 1.799651] systemd[1]: Starting Journal Service...1797lb # [ 1.855306] systemd[1]: Starting Coldplug All udev Devices...1798client # [ 1.842742] systemd-journald[72]: Collecting audit messages is disabled.1799worker1 # [ 1.790234] systemd[1]: Mounting Kernel Configuration File System...1800client # [ 1.843891] systemd[1]: Starting Coldplug All udev Devices...1801lb # [ 1.865199] systemd-journald[72]: Collecting audit messages is disabled.1802worker1 # [ 1.798371] systemd[1]: Starting Journal Service...1803client # [ 1.855698] systemd[1]: Finished Create List of Static Device Nodes.1804client # [ 1.856436] systemd[1]: Mounted Kernel Configuration File System.1805lb # [ 1.871601] systemd[1]: Finished Create List of Static Device Nodes.1806lb # [ 1.872267] systemd[1]: Mounted Kernel Configuration File System.1807worker2 # [ 1.823291] systemd[1]: Starting Load Kernel Modules...1808worker2 # [ 1.827591] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1809client # [ 1.867909] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1810worker1 # [ 1.822600] systemd[1]: Starting Load Kernel Modules...1811worker1 # [ 1.823461] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1812lb # [ 1.891486] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1813worker2 # [ 1.839820] systemd[1]: Starting Coldplug All udev Devices...1814worker2 # [ 1.845356] systemd-journald[72]: Collecting audit messages is disabled.1815lb # [ 1.900561] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1816worker1 # [ 1.850821] systemd[1]: Starting Coldplug All udev Devices...1817worker2 # [ 1.863654] systemd[1]: Finished Create List of Static Device Nodes.1818worker2 # [ 1.864364] systemd[1]: Mounted Kernel Configuration File System.1819worker1 # [ 1.853144] systemd[1]: Finished Create List of Static Device Nodes.1820client # [ 1.908242] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1821lb # [ 1.923858] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1822worker1 # [ 1.855172] systemd-journald[72]: Collecting audit messages is disabled.1823lb # [ 1.926367] systemd[1]: Starting Create Static Device Nodes in /dev...1824client # [ 1.910760] systemd[1]: Starting Create Static Device Nodes in /dev...1825worker1 # [ 1.867129] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1826worker2 # [ 1.879645] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1827worker1 # [ 1.868689] systemd[1]: Mounted Kernel Configuration File System.1828lb # [ 1.930647] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1829lb # [ 1.936813] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01830lb # [ 1.937041] [drm] features: -virgl +edid -resource_blob -host_visible1831lb # [ 1.937051] [drm] features: -context_init1832lb # [ 1.937768] [drm] number of scanouts: 11833lb # [ 1.937785] [drm] number of cap sets: 01834client # [ 1.928183] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1835client # [ 1.939863] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1836client # [ 1.948821] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01837client # [ 1.949068] [drm] features: -virgl +edid -resource_blob -host_visible1838client # [ 1.949078] [drm] features: -context_init1839client # [ 1.949775] [drm] number of scanouts: 11840client # [ 1.949794] [drm] number of cap sets: 01841lb # [ 1.967190] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1842lb # [ 1.967211] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01843worker2 # [ 1.917237] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1844lb # [ 1.975947] systemd[1]: Finished Create Static Device Nodes in /dev.1845lb # [ 1.976317] systemd[1]: Reached target Preparation for Local File Systems.1846lb # [ 1.976344] systemd[1]: Reached target Local File Systems.1847worker1 # [ 1.905384] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1848lb # [ 1.980208] systemd[1]: Starting Rule-based Manager for Device Events and Files...1849client # [ 1.968226] systemd[1]: Finished Create Static Device Nodes in /dev.1850client # [ 1.968589] systemd[1]: Reached target Preparation for Local File Systems.1851client # [ 1.968618] systemd[1]: Reached target Local File Systems.1852worker1 # [ 1.918301] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1853client # [ 1.975888] systemd[1]: Starting Rule-based Manager for Device Events and Files...1854worker1 # [ 1.927431] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01855worker1 # [ 1.927673] [drm] features: -virgl +edid -resource_blob -host_visible1856worker2 # [ 1.939993] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1857worker1 # [ 1.927684] [drm] features: -context_init1858worker1 # [ 1.928377] [drm] number of scanouts: 11859worker2 # [ 1.942539] systemd[1]: Starting Create Static Device Nodes in /dev...1860worker1 # [ 1.928411] [drm] number of cap sets: 01861client # [ 1.980038] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1862client # [ 1.980060] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01863lb # [ 1.995544] Console: switching to colour frame buffer device 160x501864worker2 # [ 1.946951] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1865worker2 # [ 1.953184] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01866worker1 # [ 1.942771] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1867worker2 # [ 1.953446] [drm] features: -virgl +edid -resource_blob -host_visible1868worker2 # [ 1.953457] [drm] features: -context_init1869worker1 # [ 1.945293] systemd[1]: Starting Create Static Device Nodes in /dev...1870worker2 # [ 1.954159] [drm] number of scanouts: 11871worker2 # [ 1.954178] [drm] number of cap sets: 01872client # [ 2.007851] Console: switching to colour frame buffer device 160x501873lb # [ 2.031344] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1874client # [ 2.014730] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1875worker1 # [ 1.958682] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1876worker1 # [ 1.958704] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01877lb # [ 2.039735] systemd[1]: Finished Load Kernel Modules.1878worker2 # [ 1.983716] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1879worker2 # [ 1.983740] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01880client # [ 2.017094] systemd-modules-load[74]: Inserted module 'dm_mod'1881client # [ 2.035822] systemd[1]: Started Journal Service.1882client # [ 2.022866] systemd-modules-load[74]: Module 'virtio_balloon' is built in1883lb # [ 2.051730] systemd[1]: Starting Apply Kernel Variables...1884worker1 # [ 1.982413] Console: switching to colour frame buffer device 160x501885lb # [ 2.042916] systemd-modules-load[74]: Inserted module 'dm_mod'1886lb # [ 2.043978] systemd-modules-load[74]: Module 'virtio_balloon' is built in1887lb # [ 2.045152] systemd-modules-load[74]: Module 'virtio_console' is built in1888lb # [ 2.046214] systemd-modules-load[74]: Inserted module 'virtio_gpu'1889worker2 # [ 2.003947] systemd[1]: Finished Create Static Device Nodes in /dev.1890worker2 # [ 2.004333] systemd[1]: Reached target Preparation for Local File Systems.1891lb # [ 2.047211] systemd-modules-load[74]: Module 'virtio_rng' is built in1892worker2 # [ 2.004365] systemd[1]: Reached target Local File Systems.1893client # [ 2.032417] systemd-modules-load[74]: Module 'virtio_console' is built in1894client # [ 2.035055] systemd-modules-load[74]: Inserted module 'virtio_gpu'1895worker2 # [ 2.008549] systemd[1]: Starting Rule-based Manager for Device Events and Files...1896lb # [ 2.069980] systemd[1]: Started Journal Service.1897worker1 # [ 2.006816] systemd[1]: Finished Create Static Device Nodes in /dev.1898client # [ 2.044294] systemd-modules-load[74]: Module 'virtio_rng' is built in1899worker1 # [ 2.007863] systemd[1]: Reached target Preparation for Local File Systems.1900client # [ 2.045358] systemd[1]: Finished Load Kernel Modules.1901worker1 # [ 2.008781] systemd[1]: Reached target Local File Systems.1902client # [ 2.046293] systemd[1]: Starting Apply Kernel Variables...1903worker2 # [ 2.015748] Console: switching to colour frame buffer device 160x501904worker2 # [ 2.022576] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1905worker1 # [ 2.010863] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1906lb # [ 2.068763] systemd-udevd[78]: Using default interface naming scheme 'v261'.1907worker1 # [ 2.018644] systemd[1]: Starting Rule-based Manager for Device Events and Files...1908worker1 # [ 2.026299] systemd[1]: Started Journal Service.1909worker2 # [ 2.041286] systemd[1]: Started Journal Service.1910client # [ 2.068126] systemd[1]: Starting Create System Files and Directories...1911lb # [ 2.085343] systemd[1]: Starting Create System Files and Directories...1912client # [ 2.072520] systemd-udevd[79]: Using default interface naming scheme 'v261'.1913lb # [ 2.088484] systemd[1]: Finished Apply Kernel Variables.1914worker1 # [ 2.024216] systemd-modules-load[74]: Inserted module 'dm_mod'1915worker1 # [ 2.025362] systemd-modules-load[74]: Module 'virtio_balloon' is built in1916worker2 # [ 2.036604] systemd-modules-load[74]: Inserted module 'dm_mod'1917worker1 # [ 2.026461] systemd-modules-load[74]: Module 'virtio_console' is built in1918worker2 # [ 2.037893] systemd-modules-load[74]: Module 'virtio_balloon' is built in1919worker1 # [ 2.027511] systemd-modules-load[74]: Inserted module 'virtio_gpu'1920worker2 # [ 2.039008] systemd-modules-load[74]: Module 'virtio_console' is built in1921worker1 # [ 2.032409] systemd-modules-load[74]: Module 'virtio_rng' is built in1922worker2 # [ 2.045811] systemd-modules-load[74]: Inserted module 'virtio_gpu'1923worker2 # [ 2.049378] systemd-modules-load[74]: Module 'virtio_rng' is built in1924worker1 # [ 2.040362] systemd[1]: Finished Load Kernel Modules.1925worker2 # [ 2.051932] systemd[1]: Starting Create System Files and Directories...1926worker1 # [ 2.042554] systemd[1]: Starting Apply Kernel Variables...1927lb # [ 2.117496] systemd[1]: Finished Create System Files and Directories.1928worker1 # [ 2.052083] systemd[1]: Starting Create System Files and Directories...1929worker2 # [ 2.065636] systemd[1]: Finished Load Kernel Modules.1930lb # [ 2.123639] systemd[1]: Started Rule-based Manager for Device Events and Files.1931client # [ 2.116105] systemd[1]: Finished Apply Kernel Variables.1932worker2 # [ 2.075685] systemd[1]: Starting Apply Kernel Variables...1933client # [ 2.124250] systemd[1]: Finished Create System Files and Directories.1934client # [ 2.129133] systemd[1]: Started Rule-based Manager for Device Events and Files.1935worker2 # [ 2.097162] systemd[1]: Finished Create System Files and Directories.1936worker1 # [ 2.097527] systemd[1]: Finished Apply Kernel Variables.1937worker2 # [ 2.109570] systemd-udevd[78]: Using default interface naming scheme 'v261'.1938worker1 # [ 2.100495] systemd-udevd[79]: Using default interface naming scheme 'v261'.1939worker2 # [ 2.111589] systemd[1]: Finished Apply Kernel Variables.1940worker1 # [ 2.113014] systemd[1]: Finished Create System Files and Directories.1941lb # [ 2.185467] systemd[1]: Starting Virtual Console Setup...1942worker2 # [ 2.140673] systemd[1]: Started Rule-based Manager for Device Events and Files.1943client # [ 2.183815] systemd[1]: Starting Virtual Console Setup...1944worker1 # [ 2.136783] systemd[1]: Started Rule-based Manager for Device Events and Files.1945lb # [ 2.232505] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1946lb # [ 2.235924] systemd[1]: Finished Virtual Console Setup.1947client # [ 2.236506] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1948worker2 # [ 2.198358] systemd[1]: Starting Virtual Console Setup...1949client # [ 2.240137] systemd[1]: Finished Virtual Console Setup.1950worker1 # [ 2.189952] systemd[1]: Starting Virtual Console Setup...1951worker2 # [ 2.248471] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1952worker2 # [ 2.251822] systemd[1]: Finished Virtual Console Setup.1953worker1 # [ 2.240516] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954worker1 # [ 2.243858] systemd[1]: Finished Virtual Console Setup.1955client # [ 2.832306] systemd[1]: Finished Coldplug All udev Devices.1956client # [ 2.837084] systemd[1]: Reached target System Initialization.1957client # [ 2.838024] systemd[1]: Reached target Basic System.1958lb # [ 2.877775] systemd[1]: Finished Coldplug All udev Devices.1959lb # [ 2.880706] systemd[1]: Reached target System Initialization.1960lb # [ 2.884147] systemd[1]: Reached target Basic System.1961worker1 # [ 2.837033] systemd[1]: Finished Coldplug All udev Devices.1962worker1 # [ 2.838412] systemd[1]: Reached target System Initialization.1963worker1 # [ 2.839233] systemd[1]: Reached target Basic System.1964worker2 # [ 2.904995] systemd[1]: Finished Coldplug All udev Devices.1965worker2 # [ 2.905964] systemd[1]: Reached target System Initialization.1966worker2 # [ 2.906763] systemd[1]: Reached target Basic System.1967lb # [ 3.028245] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1968client # [ 3.014559] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1969client # [ 3.024080] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1970client # [ 3.028164] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1971lb # [ 3.046023] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1972worker1 # [ 2.983378] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1973lb # [ 3.061883] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1974worker1 # [ 3.010603] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975worker1 # [ 3.024438] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1976client # [ 3.088807] systemd[1]: Found device /dev/disk/by-label/nixos.1977worker2 # [ 3.051987] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1978client # [ 3.095990] systemd[1]: Reached target Initrd Root Device.1979client # [ 3.101858] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1980lb # [ 3.123850] systemd[1]: Found device /dev/disk/by-label/nixos.1981lb # [ 3.132583] systemd[1]: Reached target Initrd Root Device.1982lb # [ 3.140099] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1983worker2 # [ 3.086343] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1984worker2 # [ 3.091007] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1985client # [ 3.149845] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1986client # [ 3.158138] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1987worker1 # [ 3.108987] systemd[1]: Found device /dev/disk/by-label/nixos.1988client # [ 3.160826] systemd[1]: Mounting /sysroot...1989worker1 # [ 3.110185] systemd[1]: Reached target Initrd Root Device.1990worker1 # [ 3.114546] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1991lb # [ 3.194596] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1992lb # [ 3.202511] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993lb # [ 3.206420] systemd[1]: Mounting /sysroot...1994worker2 # [ 3.158144] systemd[1]: Found device /dev/disk/by-label/nixos.1995client # [ 3.214798] EXT4-fs (vda): mounted filesystem 4e6ad68d-b89d-469d-afb4-92eda8e9d648 r/w with ordered data mode. Quota mode: none.1996worker2 # [ 3.161540] systemd[1]: Reached target Initrd Root Device.1997client # [ 3.206886] systemd[1]: Mounted /sysroot.1998worker2 # [ 3.167015] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1999client # [ 3.209250] systemd[1]: Reached target Initrd Root File System.2000worker1 # [ 3.164658] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 2,326 hidden lines…
4327worker2 # [ 53.357677] nix-daemon[862]: accepted connection from pid 963, user nix-grpc-daemon (trusted)4328worker2 # [ 53.384802] nix-daemon[862]: reaped child process 966, status = succeeded4329worker2 # [ 53.484126] nix-daemon[862]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4330worker2 # [ 53.502083] nix-daemon[862]: reaped child process 971, status = succeeded4331lb # [ 53.564151] niks3-server[1044]: 2026/09/22 13:46:31 INFO Received uploads request method=POST path=/api/pending_closures4332worker2 # [ 53.519586] nix-grpc-daemon[876]: time=2026-09-22T13:46:31.117Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4333worker2 # [ 53.521277] nix-grpc-daemon[876]: time=2026-09-22T13:46:31.117Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4334lb # [ 53.598939] niks3-server[1044]: 2026/09/22 13:46:31 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4335lb # [ 53.614448] niks3-server[1044]: 2026/09/22 13:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4336worker2 # [ 53.563829] nix-grpc-daemon[876]: time=2026-09-22T13:46:31.162Z level=INFO msg="Uploading 1 narinfos"4337lb # [ 53.626202] niks3-server[1044]: 2026/09/22 13:46:31 INFO Signed narinfos id=5 count=14338lb # [ 53.631948] niks3-server[1044]: 2026/09/22 13:46:31 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4339lb # [ 53.642508] niks3-server[1044]: 2026/09/22 13:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4340worker2 # [ 53.591548] nix-grpc-daemon[876]: time=2026-09-22T13:46:31.190Z level=INFO msg="Upload complete. (381ms)"4341lb # [ 53.649149] niks3-server[1044]: 2026/09/22 13:46:31 INFO Completed upload id=54342lb # [ 53.651968] niks3-server[1044]: 2026/09/22 13:46:31 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4343worker2 # [ 53.600298] nix-grpc-daemon[774]: ts=2026-09-22T13:46:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_in=7524344worker2 # [ 53.616900] nix-grpc-daemon[774]: ts=2026-09-22T13:46:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384345worker2 # [ 53.623858] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4346worker2 # [ 53.629606] nix-daemon[862]: reaped child process 961, status = succeeded4347client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4348worker2 # [ 53.641866] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4349client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4350client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4351client # farm-top> LOG-farm-top-t14352worker2 # [ 54.753518] nix-daemon[862]: reaped child process 975, status = succeeded4353worker2 # [ 54.822600] nix-daemon[862]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4354worker2 # [ 54.842991] nix-daemon[862]: reaped child process 994, status = succeeded4355lb # [ 54.909829] niks3-server[1044]: 2026/09/22 13:46:32 INFO Received uploads request method=POST path=/api/pending_closures4356worker2 # [ 54.866954] nix-grpc-daemon[876]: time=2026-09-22T13:46:32.463Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4357worker2 # [ 54.869280] nix-grpc-daemon[876]: time=2026-09-22T13:46:32.463Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4358lb # [ 54.955982] niks3-server[1044]: 2026/09/22 13:46:32 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4359lb # [ 54.972562] niks3-server[1044]: 2026/09/22 13:46:32 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4360worker2 # [ 54.927487] nix-grpc-daemon[876]: time=2026-09-22T13:46:32.525Z level=INFO msg="Uploading 1 narinfos"4361lb # [ 54.985167] niks3-server[1044]: 2026/09/22 13:46:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4362lb # [ 54.991479] niks3-server[1044]: 2026/09/22 13:46:32 INFO Signed narinfos id=6 count=14363lb # [ 54.996211] niks3-server[1044]: 2026/09/22 13:46:32 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4364lb # [ 55.005118] niks3-server[1044]: 2026/09/22 13:46:32 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4365lb # [ 55.010243] niks3-server[1044]: 2026/09/22 13:46:32 INFO Completed upload id=64366worker2 # [ 54.954219] nix-grpc-daemon[876]: time=2026-09-22T13:46:32.552Z level=INFO msg="Upload complete. (299ms)"4367lb # [ 55.015277] niks3-server[1044]: 2026/09/22 13:46:32 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4368worker2 # [ 54.966752] nix-grpc-daemon[774]: ts=2026-09-22T13:46:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14369worker2 # [ 54.988598] nix-daemon[862]: reaped child process 973, status = succeeded4370client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.03 seconds)4371client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4372client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.18 seconds)4373client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4374client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4375client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4376client # copying 3 paths...4377client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4378client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4379client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4380client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4381(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.52 seconds)4382subtest: repeat is answered Cached by the scheduler without building4383worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4384worker1 # finding garbage collector roots...4385worker1 # deleting specified paths...4386worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4387worker1 # deleting unused links...4388worker1 # note: hard linking is currently saving 0.0 KiB4389worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)4390worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4391worker2 # finding garbage collector roots...4392worker2 # deleting specified paths...4393worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4394worker2 # deleting unused links...4395worker2 # note: hard linking is currently saving 0.0 KiB4396worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds)4397client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24398client # don't know how to build these paths:4399client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4400worker1 # [ 56.196582] nix-grpc-daemon[765]: ts=2026-09-22T13:46:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487784401worker2 # [ 56.213459] nix-grpc-daemon[774]: ts=2026-09-22T13:46:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443384402worker2 # [ 56.222097] nix-grpc-daemon[876]: time=2026-09-22T13:46:33.817Z level=INFO msg="All 2 paths already cached"4403worker2 # [ 56.235930] nix-grpc-daemon[774]: ts=2026-09-22T13:46:33Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14404client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.28 seconds)4405client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4406client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.15 seconds)4407lb: must succeed: curl -sf localhost:9901/clusters4408lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4409lb: must succeed: curl -sf localhost:9901/clusters4410lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4411worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4412worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4413(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.18 seconds)4414subtest: inputs already in the cache are not sent to the scheduler4415lb: must succeed: curl -sf localhost:9901/clusters4416lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4417lb: must succeed: curl -sf localhost:9901/clusters4418lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4419worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4420worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4421client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24422worker2 # [ 57.100206] nix-grpc-daemon[774]: ts=2026-09-22T13:46:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384423client # don't know how to build these paths:4424client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4425worker1 # [ 57.100682] nix-grpc-daemon[765]: ts=2026-09-22T13:46:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487784426worker2 # [ 57.140184] nix-grpc-daemon[774]: ts=2026-09-22T13:46:34Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04427worker2 # [ 57.147569] nix-grpc-daemon[774]: ts=2026-09-22T13:46:34Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14428worker2 # [ 57.184661] nix-grpc-daemon[774]: ts=2026-09-22T13:46:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384429worker2 # [ 57.201213] nix-grpc-daemon[774]: ts=2026-09-22T13:46:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443384430worker2 # [ 57.216650] nix-grpc-daemon[876]: time=2026-09-22T13:46:34.815Z level=INFO msg="All 2 paths already cached"4431client # copying 1 paths to 'lb:50051'...4432client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4433worker2 # [ 57.229075] nix-grpc-daemon[774]: ts=2026-09-22T13:46:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443384434worker2 # [ 57.245163] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4435worker2 # [ 57.411250] nix-daemon[862]: accepted connection from pid 1023, user nix-grpc-daemon (trusted)4436worker2 # [ 57.437226] nix-daemon[862]: reaped child process 1026, status = succeeded4437worker2 # [ 57.542973] nix-daemon[862]: accepted connection from pid 1028, user nix-grpc-daemon (trusted)4438worker2 # [ 57.562971] nix-daemon[862]: reaped child process 1031, status = succeeded4439lb # [ 57.625185] niks3-server[1044]: 2026/09/22 13:46:35 INFO Received uploads request method=POST path=/api/pending_closures4440worker2 # [ 57.580651] nix-grpc-daemon[876]: time=2026-09-22T13:46:35.178Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4441worker2 # [ 57.582140] nix-grpc-daemon[876]: time=2026-09-22T13:46:35.178Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4442lb # [ 57.656964] niks3-server[1044]: 2026/09/22 13:46:35 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4443worker2 # [ 57.622820] nix-grpc-daemon[876]: time=2026-09-22T13:46:35.221Z level=INFO msg="Uploading 1 narinfos"4444lb # [ 57.678285] niks3-server[1044]: 2026/09/22 13:46:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4445lb # [ 57.688001] niks3-server[1044]: 2026/09/22 13:46:35 INFO Signed narinfos id=7 count=14446lb # [ 57.695203] niks3-server[1044]: 2026/09/22 13:46:35 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4447lb # [ 57.707366] niks3-server[1044]: 2026/09/22 13:46:35 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4448worker2 # [ 57.653233] nix-grpc-daemon[876]: time=2026-09-22T13:46:35.251Z level=INFO msg="Upload complete. (390ms)"4449lb # [ 57.718981] niks3-server[1044]: 2026/09/22 13:46:35 INFO Completed upload id=74450worker2 # [ 57.663277] nix-grpc-daemon[774]: ts=2026-09-22T13:46:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_in=7524451lb # [ 57.722055] niks3-server[1044]: 2026/09/22 13:46:35 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4452worker2 # [ 57.679209] nix-grpc-daemon[774]: ts=2026-09-22T13:46:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384453worker2 # [ 57.685149] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4454worker2 # [ 57.690156] nix-daemon[862]: reaped child process 1021, status = succeeded4455client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4456worker2 # [ 57.703613] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4457client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4458client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4459client # farm-top> LOG-farm-top-t1b4460worker2 # [ 58.595888] nix-daemon[862]: reaped child process 1035, status = succeeded4461worker2 # [ 58.685414] nix-daemon[862]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4462worker2 # [ 58.709023] nix-daemon[862]: reaped child process 1054, status = succeeded4463lb # [ 58.763865] niks3-server[1044]: 2026/09/22 13:46:36 INFO Received uploads request method=POST path=/api/pending_closures4464worker2 # [ 58.720429] nix-grpc-daemon[876]: time=2026-09-22T13:46:36.317Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4465worker2 # [ 58.721900] nix-grpc-daemon[876]: time=2026-09-22T13:46:36.317Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4466lb # [ 58.817283] niks3-server[1044]: 2026/09/22 13:46:36 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4467lb # [ 58.827278] niks3-server[1044]: 2026/09/22 13:46:36 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4468worker2 # [ 58.779879] nix-grpc-daemon[876]: time=2026-09-22T13:46:36.378Z level=INFO msg="Uploading 1 narinfos"4469lb # [ 58.838978] niks3-server[1044]: 2026/09/22 13:46:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4470lb # [ 58.846214] niks3-server[1044]: 2026/09/22 13:46:36 INFO Signed narinfos id=8 count=14471lb # [ 58.853310] niks3-server[1044]: 2026/09/22 13:46:36 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4472lb # [ 58.859166] niks3-server[1044]: 2026/09/22 13:46:36 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4473lb # [ 58.862060] niks3-server[1044]: 2026/09/22 13:46:36 INFO Completed upload id=84474lb # [ 58.864983] niks3-server[1044]: 2026/09/22 13:46:36 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4475worker2 # [ 58.800696] nix-grpc-daemon[876]: time=2026-09-22T13:46:36.399Z level=INFO msg="Upload complete. (287ms)"4476worker2 # [ 58.814362] nix-grpc-daemon[774]: ts=2026-09-22T13:46:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14477worker2 # [ 58.834572] nix-daemon[862]: reaped child process 1033, status = succeeded4478client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.98 seconds)4479client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4480client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds)4481lb: must succeed: curl -sf localhost:9901/clusters4482lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4483lb: must succeed: curl -sf localhost:9901/clusters4484lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4485worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4486worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4487(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.54 seconds)4488subtest: outputs read back through any worker carry the cache signature4489client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24490worker2 # [ 59.453551] nix-grpc-daemon[774]: ts=2026-09-22T13:46:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384491client # don't know how to build these paths:4492client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4493worker1 # [ 59.457951] nix-grpc-daemon[765]: ts=2026-09-22T13:46:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487784494worker2 # [ 59.492151] nix-grpc-daemon[774]: ts=2026-09-22T13:46:37Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04495worker2 # [ 59.498796] nix-grpc-daemon[774]: ts=2026-09-22T13:46:37Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=14496worker1 # [ 59.491230] nix-grpc-daemon[765]: ts=2026-09-22T13:46:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784497worker1 # [ 59.508132] nix-grpc-daemon[765]: ts=2026-09-22T13:46:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487784498client # copying 1 paths to 'lb:50051'...4499client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4500worker1 # [ 59.524847] nix-grpc-daemon[765]: ts=2026-09-22T13:46:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487784501worker1 # [ 59.541012] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4502worker1 # [ 59.568843] nix-grpc-daemon[765]: ts=2026-09-22T13:46:37Z level=info event=niks3_push_started pid=8584503worker1 # [ 59.738427] nix-grpc-daemon[858]: time=2026-09-22T13:46:37.350Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4504worker1 # [ 59.854024] nix-daemon[833]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4505worker1 # [ 59.879663] nix-daemon[833]: reaped child process 874, status = succeeded4506worker1 # [ 59.979199] nix-daemon[833]: accepted connection from pid 876, user nix-grpc-daemon (trusted)4507worker1 # [ 59.997088] nix-daemon[833]: reaped child process 879, status = succeeded4508lb # [ 60.070259] niks3-server[1044]: 2026/09/22 13:46:37 INFO Received uploads request method=POST path=/api/pending_closures4509worker1 # [ 60.014950] nix-grpc-daemon[858]: time=2026-09-22T13:46:37.626Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4510worker1 # [ 60.016872] nix-grpc-daemon[858]: time=2026-09-22T13:46:37.626Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4511lb # [ 60.090169] envoy[913]: conn peer=[2001:db8:1::1]:52132 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9532 tx=4420 ms=58394512lb # [ 60.101817] envoy[913]: conn peer=[2001:db8:1::1]:37448 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=524513lb # [ 60.112735] envoy[913]: conn peer=[2001:db8:1::1]:37456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1921 ms=17274514lb # [ 60.129686] niks3-server[1044]: 2026/09/22 13:46:37 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4515worker1 # [ 60.072548] nix-grpc-daemon[858]: time=2026-09-22T13:46:37.684Z level=INFO msg="Uploading 1 narinfos"4516lb # [ 60.139419] niks3-server[1044]: 2026/09/22 13:46:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4517lb # [ 60.149392] niks3-server[1044]: 2026/09/22 13:46:37 INFO Signed narinfos id=9 count=14518lb # [ 60.151547] niks3-server[1044]: 2026/09/22 13:46:37 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4519lb # [ 60.161126] niks3-server[1044]: 2026/09/22 13:46:37 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4520lb # [ 60.171692] niks3-server[1044]: 2026/09/22 13:46:37 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4521lb # [ 60.175887] niks3-server[1044]: 2026/09/22 13:46:37 INFO Completed upload id=94522worker1 # [ 60.109782] nix-grpc-daemon[858]: time=2026-09-22T13:46:37.722Z level=INFO msg="Upload complete. (367ms)"4523worker1 # [ 60.121195] nix-grpc-daemon[765]: ts=2026-09-22T13:46:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_in=5124524worker1 # [ 60.139120] nix-grpc-daemon[765]: ts=2026-09-22T13:46:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784525worker1 # [ 60.148241] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4526worker1 # [ 60.156926] nix-daemon[833]: reaped child process 856, status = succeeded4527client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4528worker1 # [ 60.163933] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4529client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4530client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4531client # farm-a> LOG-farm-a-sig4532worker1 # [ 61.665159] nix-daemon[833]: reaped child process 883, status = succeeded4533worker1 # [ 61.736675] nix-daemon[833]: accepted connection from pid 899, user nix-grpc-daemon (trusted)4534worker1 # [ 61.757703] nix-daemon[833]: reaped child process 902, status = succeeded4535lb # [ 61.827314] niks3-server[1044]: 2026/09/22 13:46:39 INFO Received uploads request method=POST path=/api/pending_closures4536worker1 # [ 61.772432] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.381Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4537worker1 # [ 61.774196] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.381Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4538lb # [ 61.867540] niks3-server[1044]: 2026/09/22 13:46:39 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4539worker1 # [ 61.815485] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.426Z level=INFO msg="Uploading 1 narinfos"4540lb # [ 61.880907] niks3-server[1044]: 2026/09/22 13:46:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4541lb # [ 61.893208] niks3-server[1044]: 2026/09/22 13:46:39 INFO Signed narinfos id=10 count=14542lb # [ 61.898820] niks3-server[1044]: 2026/09/22 13:46:39 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4543lb # [ 61.908417] niks3-server[1044]: 2026/09/22 13:46:39 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4544lb # [ 61.915280] niks3-server[1044]: 2026/09/22 13:46:39 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4545lb # [ 61.921017] niks3-server[1044]: 2026/09/22 13:46:39 INFO Completed upload id=104546worker1 # [ 61.854258] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.466Z level=INFO msg="Upload complete. (280ms)"4547lb # [ 61.925891] niks3-server[1044]: 2026/09/22 13:46:39 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4548worker2 # [ 61.884409] nix-grpc-daemon[774]: ts=2026-09-22T13:46:39Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04549worker1 # [ 61.865120] nix-grpc-daemon[765]: ts=2026-09-22T13:46:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14550worker2 # [ 61.884799] nix-grpc-daemon[774]: ts=2026-09-22T13:46:39Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14551worker1 # [ 61.885360] nix-daemon[833]: reaped child process 881, status = succeeded4552worker1 # [ 61.918497] nix-grpc-daemon[765]: ts=2026-09-22T13:46:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784553worker1 # [ 61.935077] nix-grpc-daemon[765]: ts=2026-09-22T13:46:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487784554worker1 # [ 61.949156] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.561Z level=INFO msg="All 1 paths already cached"4555client # copying 1 paths to 'lb:50051'...4556client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4557worker1 # [ 61.959865] nix-grpc-daemon[765]: ts=2026-09-22T13:46:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487784558worker1 # [ 61.974056] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4559worker1 # [ 62.127100] nix-daemon[833]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4560worker1 # [ 62.147504] nix-daemon[833]: reaped child process 909, status = succeeded4561worker1 # [ 62.252670] nix-daemon[833]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4562worker1 # [ 62.274442] nix-daemon[833]: reaped child process 914, status = succeeded4563lb # [ 62.338931] niks3-server[1044]: 2026/09/22 13:46:39 INFO Received uploads request method=POST path=/api/pending_closures4564worker1 # [ 62.282354] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.894Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4565worker1 # [ 62.283883] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.894Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4566lb # [ 62.368228] niks3-server[1044]: 2026/09/22 13:46:39 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4567lb # [ 62.386850] niks3-server[1044]: 2026/09/22 13:46:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4568worker1 # [ 62.325678] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.937Z level=INFO msg="Uploading 1 narinfos"4569lb # [ 62.398713] niks3-server[1044]: 2026/09/22 13:46:40 INFO Signed narinfos id=11 count=14570lb # [ 62.407015] niks3-server[1044]: 2026/09/22 13:46:40 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4571lb # [ 62.420042] niks3-server[1044]: 2026/09/22 13:46:40 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4572lb # [ 62.428120] niks3-server[1044]: 2026/09/22 13:46:40 INFO Completed upload id=114573worker1 # [ 62.361119] nix-grpc-daemon[858]: time=2026-09-22T13:46:39.973Z level=INFO msg="Upload complete. (370ms)"4574lb # [ 62.431541] niks3-server[1044]: 2026/09/22 13:46:40 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4575worker1 # [ 62.373234] nix-grpc-daemon[765]: ts=2026-09-22T13:46:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_in=6244576worker1 # [ 62.384392] nix-grpc-daemon[765]: ts=2026-09-22T13:46:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784577worker1 # [ 62.391734] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4578worker1 # [ 62.398759] nix-daemon[833]: reaped child process 904, status = succeeded4579client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4580worker1 # [ 62.411704] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4581client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4582client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4583client # farm-b> LOG-farm-b-sig4584worker1 # [ 63.684522] nix-daemon[833]: reaped child process 918, status = succeeded4585worker1 # [ 63.802529] nix-daemon[833]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4586worker1 # [ 63.824087] nix-daemon[833]: reaped child process 937, status = succeeded4587lb # [ 63.890164] niks3-server[1044]: 2026/09/22 13:46:41 INFO Received uploads request method=POST path=/api/pending_closures4588worker1 # [ 63.835789] nix-grpc-daemon[858]: time=2026-09-22T13:46:41.446Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4589worker1 # [ 63.838142] nix-grpc-daemon[858]: time=2026-09-22T13:46:41.447Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4590lb # [ 63.924470] niks3-server[1044]: 2026/09/22 13:46:41 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4591lb # [ 63.954657] niks3-server[1044]: 2026/09/22 13:46:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4592worker1 # [ 63.893875] nix-grpc-daemon[858]: time=2026-09-22T13:46:41.505Z level=INFO msg="Uploading 1 narinfos"4593lb # [ 63.963477] niks3-server[1044]: 2026/09/22 13:46:41 INFO Signed narinfos id=12 count=14594lb # [ 63.974781] niks3-server[1044]: 2026/09/22 13:46:41 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4595lb # [ 63.985228] niks3-server[1044]: 2026/09/22 13:46:41 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4596lb # [ 63.993998] niks3-server[1044]: 2026/09/22 13:46:41 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4597lb # [ 64.000747] niks3-server[1044]: 2026/09/22 13:46:41 INFO Completed upload id=124598worker1 # [ 63.934537] nix-grpc-daemon[858]: time=2026-09-22T13:46:41.546Z level=INFO msg="Upload complete. (310ms)"4599lb # [ 64.005611] niks3-server[1044]: 2026/09/22 13:46:41 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4600worker2 # [ 63.966884] nix-grpc-daemon[774]: ts=2026-09-22T13:46:41Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04601worker2 # [ 63.972757] nix-grpc-daemon[774]: ts=2026-09-22T13:46:41Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14602worker1 # [ 63.949351] nix-grpc-daemon[765]: ts=2026-09-22T13:46:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14603worker1 # [ 63.968765] nix-daemon[833]: reaped child process 916, status = succeeded4604worker1 # [ 64.002297] nix-grpc-daemon[765]: ts=2026-09-22T13:46:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784605worker1 # [ 64.019209] nix-grpc-daemon[765]: ts=2026-09-22T13:46:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487784606worker1 # [ 64.035629] nix-grpc-daemon[858]: time=2026-09-22T13:46:41.647Z level=INFO msg="All 2 paths already cached"4607client # copying 1 paths to 'lb:50051'...4608client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4609worker1 # [ 64.047517] nix-grpc-daemon[765]: ts=2026-09-22T13:46:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487784610worker1 # [ 64.066000] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4611worker1 # [ 64.226374] nix-daemon[833]: accepted connection from pid 941, user nix-grpc-daemon (trusted)4612worker1 # [ 64.256099] nix-daemon[833]: reaped child process 944, status = succeeded4613worker1 # [ 64.350437] nix-daemon[833]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4614worker1 # [ 64.367812] nix-daemon[833]: reaped child process 949, status = succeeded4615lb # [ 64.440318] niks3-server[1044]: 2026/09/22 13:46:42 INFO Received uploads request method=POST path=/api/pending_closures4616worker1 # [ 64.383610] nix-grpc-daemon[858]: time=2026-09-22T13:46:41.995Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4617worker1 # [ 64.385266] nix-grpc-daemon[858]: time=2026-09-22T13:46:41.995Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4618lb # [ 64.472359] niks3-server[1044]: 2026/09/22 13:46:42 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4619worker1 # [ 64.421005] nix-grpc-daemon[858]: time=2026-09-22T13:46:42.033Z level=INFO msg="Uploading 1 narinfos"4620lb # [ 64.487210] niks3-server[1044]: 2026/09/22 13:46:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4621lb # [ 64.497832] niks3-server[1044]: 2026/09/22 13:46:42 INFO Signed narinfos id=13 count=14622lb # [ 64.504560] niks3-server[1044]: 2026/09/22 13:46:42 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4623lb # [ 64.520190] niks3-server[1044]: 2026/09/22 13:46:42 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4624lb # [ 64.527698] niks3-server[1044]: 2026/09/22 13:46:42 INFO Completed upload id=134625lb # [ 64.532638] niks3-server[1044]: 2026/09/22 13:46:42 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4626worker1 # [ 64.461458] nix-grpc-daemon[858]: time=2026-09-22T13:46:42.073Z level=INFO msg="Upload complete. (377ms)"4627worker1 # [ 64.472492] nix-grpc-daemon[765]: ts=2026-09-22T13:46:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_in=7524628worker1 # [ 64.488264] nix-grpc-daemon[765]: ts=2026-09-22T13:46:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784629worker1 # [ 64.494827] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4630worker1 # [ 64.500208] nix-daemon[833]: reaped child process 939, status = succeeded4631client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4632worker1 # [ 64.514044] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4633client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4634client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4635client # farm-top> LOG-farm-top-sig4636worker1 # [ 65.972372] nix-daemon[833]: reaped child process 953, status = succeeded4637worker1 # [ 66.067878] nix-daemon[833]: accepted connection from pid 969, user nix-grpc-daemon (trusted)4638worker1 # [ 66.084533] nix-daemon[833]: reaped child process 972, status = succeeded4639lb # [ 66.162042] niks3-server[1044]: 2026/09/22 13:46:43 INFO Received uploads request method=POST path=/api/pending_closures4640worker1 # [ 66.106938] nix-grpc-daemon[858]: time=2026-09-22T13:46:43.717Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4641worker1 # [ 66.108656] nix-grpc-daemon[858]: time=2026-09-22T13:46:43.717Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4642lb # [ 66.200591] niks3-server[1044]: 2026/09/22 13:46:43 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4643lb # [ 66.215488] niks3-server[1044]: 2026/09/22 13:46:43 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4644worker1 # [ 66.163055] nix-grpc-daemon[858]: time=2026-09-22T13:46:43.775Z level=INFO msg="Uploading 1 narinfos"4645lb # [ 66.229301] niks3-server[1044]: 2026/09/22 13:46:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4646lb # [ 66.242504] niks3-server[1044]: 2026/09/22 13:46:43 INFO Signed narinfos id=14 count=14647lb # [ 66.248903] niks3-server[1044]: 2026/09/22 13:46:43 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4648lb # [ 66.264250] niks3-server[1044]: 2026/09/22 13:46:43 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4649lb # [ 66.273932] niks3-server[1044]: 2026/09/22 13:46:43 INFO Completed upload id=144650worker1 # [ 66.206292] nix-grpc-daemon[858]: time=2026-09-22T13:46:43.818Z level=INFO msg="Upload complete. (281ms)"4651lb # [ 66.276727] niks3-server[1044]: 2026/09/22 13:46:43 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4652worker1 # [ 66.221263] nix-grpc-daemon[765]: ts=2026-09-22T13:46:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14653worker1 # [ 66.238342] nix-daemon[833]: reaped child process 951, status = succeeded4654client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 7.04 seconds)4655client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4656client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4657client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4658worker1 # [ 66.592647] nix-grpc-daemon[765]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:445084659worker1 # [ 66.635353] nix-grpc-daemon[765]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:445084660client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds)4661client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4662worker2 # [ 66.899660] nix-grpc-daemon[774]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:464924663client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4664client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4665client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4666client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4667worker2 # [ 66.928264] nix-grpc-daemon[774]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:464924668worker2 # [ 66.937592] nix-grpc-daemon[774]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4669worker2 # [ 66.957372] nix-grpc-daemon[774]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4670worker2 # [ 66.975157] nix-grpc-daemon[774]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4671client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.31 seconds)4672client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4673client # finding garbage collector roots...4674client # removing stale temporary roots file "/nix/var/nix/temproots/858"4675client # removing stale temporary roots file "/nix/var/nix/temproots/837"4676client # removing stale temporary roots file "/nix/var/nix/temproots/881"4677client # removing stale temporary roots file "/nix/var/nix/temproots/804"4678client # removing stale temporary roots file "/nix/var/nix/temproots/789"4679client # removing stale temporary roots file "/nix/var/nix/temproots/825"4680client # removing stale temporary roots file "/nix/var/nix/temproots/866"4681client # removing stale temporary roots file "/nix/var/nix/temproots/845"4682client # deleting specified paths...4683client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4684client # deleting unused links...4685client # note: hard linking is currently saving 0.0 KiB4686worker2 # [ 67.353787] nix-grpc-daemon[774]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384687worker2 # [ 67.364513] nix-grpc-daemon[774]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443384688worker2 # [ 67.373090] nix-grpc-daemon[876]: time=2026-09-22T13:46:44.969Z level=INFO msg="All 1 paths already cached"4689worker2 # [ 67.379934] nix-grpc-daemon[774]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443384690worker1 # [ 67.374928] nix-grpc-daemon[765]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487784691worker1 # [ 67.381618] nix-grpc-daemon[765]: ts=2026-09-22T13:46:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487784692client # copying 3 paths...4693worker2 # [ 67.406643] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443384694client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4695worker2 # [ 67.435354] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443384696worker2 # [ 67.443131] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_out=2324697worker1 # [ 67.443899] nix-grpc-daemon[765]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:487784698worker1 # [ 67.452188] nix-grpc-daemon[765]: ts=2026-09-22T13:46:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_out=1764699worker1 # [ 67.461455] nix-grpc-daemon[765]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:487784700worker1 # [ 67.466715] nix-grpc-daemon[765]: ts=2026-09-22T13:46:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_out=1284701client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4702client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4703client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.54 seconds)4704(finished: subtest: outputs read back through any worker carry the cache signature, in 8.33 seconds)4705subtest: two clients wanting the same drv build it once4706client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4707client # [ 67.659766] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4708client # Running as unit: dedup1.service; invocation ID: 46b2744a510244ca962402c8ee17089f4709client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4710client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4711worker2 # [ 67.812406] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384712client # [ 67.861293] nix[934]: don't know how to build these paths:4713client # [ 67.862410] nix[934]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4714worker2 # [ 67.829957] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04715worker2 # [ 67.838134] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14716worker2 # [ 67.877059] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384717worker2 # [ 67.893370] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443384718client # [ 67.948405] nix[934]: copying 1 paths to 'lb:50051'...4719worker2 # [ 67.910517] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443384720client # [ 67.952946] nix[934]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4721worker2 # [ 67.926759] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4722worker2 # [ 68.085284] nix-daemon[862]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4723worker2 # [ 68.106046] nix-daemon[862]: reaped child process 1077, status = succeeded4724worker2 # [ 68.213125] nix-daemon[862]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4725worker2 # [ 68.229994] nix-daemon[862]: reaped child process 1082, status = succeeded4726lb # [ 68.294047] niks3-server[1044]: 2026/09/22 13:46:45 INFO Received uploads request method=POST path=/api/pending_closures4727worker2 # [ 68.250145] nix-grpc-daemon[876]: time=2026-09-22T13:46:45.848Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4728worker2 # [ 68.251754] nix-grpc-daemon[876]: time=2026-09-22T13:46:45.848Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4729lb # [ 68.330460] niks3-server[1044]: 2026/09/22 13:46:45 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4730worker2 # [ 68.299201] nix-grpc-daemon[876]: time=2026-09-22T13:46:45.897Z level=INFO msg="Uploading 1 narinfos"4731lb # [ 68.346125] niks3-server[1044]: 2026/09/22 13:46:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4732lb # [ 68.358385] niks3-server[1044]: 2026/09/22 13:46:45 INFO Signed narinfos id=15 count=14733lb # [ 68.365456] niks3-server[1044]: 2026/09/22 13:46:45 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4734lb # [ 68.379394] niks3-server[1044]: 2026/09/22 13:46:45 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4735lb # [ 68.386143] niks3-server[1044]: 2026/09/22 13:46:45 INFO Completed upload id=154736worker2 # [ 68.330130] nix-grpc-daemon[876]: time=2026-09-22T13:46:45.928Z level=INFO msg="Upload complete. (386ms)"4737lb # [ 68.389154] niks3-server[1044]: 2026/09/22 13:46:46 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4738worker2 # [ 68.341313] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_in=4324739worker2 # [ 68.356576] nix-grpc-daemon[774]: ts=2026-09-22T13:46:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384740worker2 # [ 68.363734] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4741worker2 # [ 68.369672] nix-daemon[862]: reaped child process 1072, status = succeeded4742client # [ 68.420664] nix[934]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4743worker2 # [ 68.383729] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4744client # [ 68.757423] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4745client # Running as unit: dedup2.service; invocation ID: 0333d617ea41421da6ef6e64e61aa5de4746client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4747worker2 # [ 68.934973] nix-grpc-daemon[774]: ts=2026-09-22T13:46:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384748client # [ 68.990087] nix[950]: this derivation will be built:4749worker2 # [ 68.950816] nix-grpc-daemon[774]: ts=2026-09-22T13:46:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443384750client # [ 69.002340] nix[950]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4751worker2 # [ 68.970427] nix-grpc-daemon[774]: ts=2026-09-22T13:46:46Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04752client # [ 69.019080] nix[950]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4753worker2 # [ 68.977854] nix-grpc-daemon[774]: ts=2026-09-22T13:46:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384754client: must succeed: sleep 24755lb # [ 70.090365] envoy[913]: conn peer=[2001:db8:1::1]:37462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9558 tx=4386 ms=67884756lb # [ 70.095467] envoy[913]: conn peer=[2001:db8:1::1]:58946 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2931 ms=1924757lb # [ 70.101357] envoy[913]: conn peer=[2001:db8:1::1]:58976 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=804758lb # [ 70.106194] envoy[913]: conn peer=[2001:db8:1::1]:58960 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=954759lb # [ 70.110963] envoy[913]: conn peer=[2001:db8:1::1]:58954 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=1154760client: (finished: must succeed: sleep 2, in 2.05 seconds)4761client: waiting for success: ! systemctl is-active dedup1 dedup24762worker2 # [ 72.495399] nix-daemon[862]: reaped child process 1086, status = succeeded4763worker2 # [ 72.566293] nix-daemon[862]: accepted connection from pid 1118, user nix-grpc-daemon (trusted)4764worker2 # [ 72.586675] nix-daemon[862]: reaped child process 1121, status = succeeded4765lb # [ 72.649106] niks3-server[1044]: 2026/09/22 13:46:50 INFO Received uploads request method=POST path=/api/pending_closures4766worker2 # [ 72.604477] nix-grpc-daemon[876]: time=2026-09-22T13:46:50.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4767worker2 # [ 72.606324] nix-grpc-daemon[876]: time=2026-09-22T13:46:50.200Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4768lb # [ 72.696520] niks3-server[1044]: 2026/09/22 13:46:50 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4769worker2 # [ 72.668483] nix-grpc-daemon[876]: time=2026-09-22T13:46:50.266Z level=INFO msg="Uploading 1 narinfos"4770lb # [ 72.718300] niks3-server[1044]: 2026/09/22 13:46:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4771lb # [ 72.729560] niks3-server[1044]: 2026/09/22 13:46:50 INFO Signed narinfos id=16 count=14772lb # [ 72.731783] niks3-server[1044]: 2026/09/22 13:46:50 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4773lb # [ 72.743921] niks3-server[1044]: 2026/09/22 13:46:50 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4774lb # [ 72.752724] niks3-server[1044]: 2026/09/22 13:46:50 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4775worker2 # [ 72.704249] nix-grpc-daemon[876]: time=2026-09-22T13:46:50.302Z level=INFO msg="Upload complete. (306ms)"4776lb # [ 72.762859] niks3-server[1044]: 2026/09/22 13:46:50 INFO Completed upload id=164777lb # [ 72.766250] niks3-server[1044]: 2026/09/22 13:46:50 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4778worker2 # [ 72.718257] nix-grpc-daemon[774]: ts=2026-09-22T13:46:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14779worker2 # [ 72.740140] nix-grpc-daemon[774]: ts=2026-09-22T13:46:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14780worker2 # [ 72.747955] nix-daemon[862]: reaped child process 1084, status = succeeded4781client # [ 72.819404] systemd[1]: dedup2.service: Deactivated successfully.4782client # [ 72.823609] systemd[1]: dedup2.service: Consumed 205ms CPU time over 4.063s wall clock time, 35.2M memory peak, 6.6K incoming IP traffic, 6.1K outgoing IP traffic.4783client # [ 72.830851] systemd[1]: dedup1.service: Deactivated successfully.4784client # [ 72.834478] systemd[1]: dedup1.service: Consumed 175ms CPU time over 5.171s wall clock time, 39.5M memory peak, 7.5K incoming IP traffic, 8.7K outgoing IP traffic.4785client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4786client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4787client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4788worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4789worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4790worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4791worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4792worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4793worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4794worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4795worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4796(finished: subtest: two clients wanting the same drv build it once, in 6.09 seconds)4797subtest: --timeout and --max-silent-time reach the builder4798worker2 # [ 73.880799] nix-grpc-daemon[774]: ts=2026-09-22T13:46:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384799worker2 # [ 73.893923] nix-grpc-daemon[774]: ts=2026-09-22T13:46:51Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04800worker2 # [ 73.900340] nix-grpc-daemon[774]: ts=2026-09-22T13:46:51Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14801worker1 # [ 73.894206] nix-grpc-daemon[765]: ts=2026-09-22T13:46:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784802worker1 # [ 73.911289] nix-grpc-daemon[765]: ts=2026-09-22T13:46:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487784803worker1 # [ 73.926174] nix-grpc-daemon[765]: ts=2026-09-22T13:46:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487784804worker1 # [ 73.941069] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4805worker1 # [ 74.105351] nix-daemon[833]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4806worker1 # [ 74.121946] nix-daemon[833]: reaped child process 1000, status = succeeded4807worker1 # [ 74.226649] nix-daemon[833]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4808worker1 # [ 74.242140] nix-daemon[833]: reaped child process 1005, status = succeeded4809lb # [ 74.318223] niks3-server[1044]: 2026/09/22 13:46:51 INFO Received uploads request method=POST path=/api/pending_closures4810worker1 # [ 74.262358] nix-grpc-daemon[858]: time=2026-09-22T13:46:51.874Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4811worker1 # [ 74.263845] nix-grpc-daemon[858]: time=2026-09-22T13:46:51.874Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4812lb # [ 74.348529] niks3-server[1044]: 2026/09/22 13:46:51 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4813worker1 # [ 74.295605] nix-grpc-daemon[858]: time=2026-09-22T13:46:51.907Z level=INFO msg="Uploading 1 narinfos"4814lb # [ 74.362065] niks3-server[1044]: 2026/09/22 13:46:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4815lb # [ 74.372408] niks3-server[1044]: 2026/09/22 13:46:51 INFO Signed narinfos id=17 count=14816lb # [ 74.379913] niks3-server[1044]: 2026/09/22 13:46:51 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4817lb # [ 74.393191] niks3-server[1044]: 2026/09/22 13:46:52 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4818worker1 # [ 74.332565] nix-grpc-daemon[858]: time=2026-09-22T13:46:51.944Z level=INFO msg="Upload complete. (374ms)"4819lb # [ 74.402296] niks3-server[1044]: 2026/09/22 13:46:52 INFO Completed upload id=174820worker1 # [ 74.340316] nix-grpc-daemon[765]: ts=2026-09-22T13:46:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_in=4244821lb # [ 74.407686] niks3-server[1044]: 2026/09/22 13:46:52 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4822worker1 # [ 74.355953] nix-grpc-daemon[765]: ts=2026-09-22T13:46:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784823worker1 # [ 74.363406] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4824worker1 # [ 74.369044] nix-daemon[833]: reaped child process 995, status = succeeded4825worker1 # [ 74.380898] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4826lb # [ 80.094171] envoy[913]: conn peer=[2001:db8:1::1]:58994 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1598 ms=38024827worker1 # [ 80.605757] nix-grpc-daemon[765]: ts=2026-09-22T13:46:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04828worker1 # [ 80.659202] nix-daemon[833]: reaped child process 1007, status = succeeded4829lb # [ 80.099585] envoy[913]: conn peer=[2001:db8:1::1]:58988 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3035 tx=1558 ms=49374830worker1 # [ 80.662217] nix-daemon[833]: reaped child process 1009, status = succeeded4831worker2 # [ 80.898847] nix-grpc-daemon[774]: ts=2026-09-22T13:46:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384832worker2 # [ 80.930548] nix-grpc-daemon[774]: ts=2026-09-22T13:46:58Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04833worker2 # [ 80.938829] nix-grpc-daemon[774]: ts=2026-09-22T13:46:58Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14834worker2 # [ 80.949181] nix-grpc-daemon[774]: ts=2026-09-22T13:46:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384835worker2 # [ 80.979371] nix-grpc-daemon[774]: ts=2026-09-22T13:46:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443384836worker2 # [ 81.001161] nix-grpc-daemon[774]: ts=2026-09-22T13:46:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443384837worker2 # [ 81.022654] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4838worker2 # [ 81.236999] nix-daemon[862]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)4839worker2 # [ 81.265770] nix-daemon[862]: reaped child process 1141, status = succeeded4840worker2 # [ 81.393722] nix-daemon[862]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4841worker2 # [ 81.417670] nix-daemon[862]: reaped child process 1146, status = succeeded4842lb # [ 81.479972] niks3-server[1044]: 2026/09/22 13:46:59 INFO Received uploads request method=POST path=/api/pending_closures4843worker2 # [ 81.439953] nix-grpc-daemon[876]: time=2026-09-22T13:46:59.037Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4844worker2 # [ 81.441721] nix-grpc-daemon[876]: time=2026-09-22T13:46:59.037Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4845lb # [ 81.561945] niks3-server[1044]: 2026/09/22 13:46:59 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4846lb # [ 81.592396] niks3-server[1044]: 2026/09/22 13:46:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4847lb # [ 81.600936] niks3-server[1044]: 2026/09/22 13:46:59 INFO Signed narinfos id=18 count=14848worker2 # [ 81.544145] nix-grpc-daemon[876]: time=2026-09-22T13:46:59.141Z level=INFO msg="Uploading 1 narinfos"4849lb # [ 81.606601] niks3-server[1044]: 2026/09/22 13:46:59 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4850lb # [ 81.623016] niks3-server[1044]: 2026/09/22 13:46:59 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4851lb # [ 81.635299] niks3-server[1044]: 2026/09/22 13:46:59 INFO Completed upload id=184852worker2 # [ 81.579354] nix-grpc-daemon[876]: time=2026-09-22T13:46:59.177Z level=INFO msg="Upload complete. (528ms)"4853lb # [ 81.638887] niks3-server[1044]: 2026/09/22 13:46:59 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4854worker2 # [ 81.593500] nix-grpc-daemon[774]: ts=2026-09-22T13:46:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_in=4244855worker2 # [ 81.609638] nix-grpc-daemon[774]: ts=2026-09-22T13:46:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384856worker2 # [ 81.626130] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4857worker2 # [ 81.637004] nix-daemon[862]: reaped child process 1136, status = succeeded4858worker2 # [ 81.653652] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4859worker2 # [ 87.675132] nix-grpc-daemon[774]: ts=2026-09-22T13:47:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04860worker2 # [ 87.692683] nix-daemon[862]: reaped child process 1148, status = succeeded4861worker2 # [ 87.695034] nix-daemon[862]: reaped child process 1150, status = succeeded4862(finished: subtest: --timeout and --max-silent-time reach the builder, in 14.25 seconds)4863subtest: the first client leaving does not fail the second4864client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4865client # [ 88.017230] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4866client # Running as unit: share1.service; invocation ID: 099dd11fc6014949b8ca66fa28dc5f7b4867client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4868client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4869client # [ 88.226986] nix[1023]: don't know how to build these paths:4870client # [ 88.228106] nix[1023]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4871worker2 # [ 88.185138] nix-grpc-daemon[774]: ts=2026-09-22T13:47:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384872worker2 # [ 88.199183] nix-grpc-daemon[774]: ts=2026-09-22T13:47:05Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04873worker1 # [ 88.194094] nix-grpc-daemon[765]: ts=2026-09-22T13:47:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784874worker2 # [ 88.205211] nix-grpc-daemon[774]: ts=2026-09-22T13:47:05Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14875worker1 # [ 88.211285] nix-grpc-daemon[765]: ts=2026-09-22T13:47:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487784876client # [ 88.275299] nix[1023]: copying 1 paths to 'lb:50051'...4877client # [ 88.278829] nix[1023]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4878worker1 # [ 88.228042] nix-grpc-daemon[765]: ts=2026-09-22T13:47:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487784879worker1 # [ 88.243566] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4880worker1 # [ 88.413626] nix-daemon[833]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4881worker1 # [ 88.435322] nix-daemon[833]: reaped child process 1035, status = succeeded4882worker1 # [ 88.535814] nix-daemon[833]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)4883worker1 # [ 88.556396] nix-daemon[833]: reaped child process 1040, status = succeeded4884lb # [ 88.630068] niks3-server[1044]: 2026/09/22 13:47:06 INFO Received uploads request method=POST path=/api/pending_closures4885worker1 # [ 88.572376] nix-grpc-daemon[858]: time=2026-09-22T13:47:06.184Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4886worker1 # [ 88.573900] nix-grpc-daemon[858]: time=2026-09-22T13:47:06.184Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4887lb # [ 88.668322] niks3-server[1044]: 2026/09/22 13:47:06 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4888lb # [ 88.689377] niks3-server[1044]: 2026/09/22 13:47:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4889worker1 # [ 88.628796] nix-grpc-daemon[858]: time=2026-09-22T13:47:06.240Z level=INFO msg="Uploading 1 narinfos"4890lb # [ 88.700966] niks3-server[1044]: 2026/09/22 13:47:06 INFO Signed narinfos id=19 count=14891lb # [ 88.703716] niks3-server[1044]: 2026/09/22 13:47:06 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4892lb # [ 88.719370] niks3-server[1044]: 2026/09/22 13:47:06 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4893lb # [ 88.727769] niks3-server[1044]: 2026/09/22 13:47:06 INFO Completed upload id=194894worker1 # [ 88.660196] nix-grpc-daemon[858]: time=2026-09-22T13:47:06.272Z level=INFO msg="Upload complete. (395ms)"4895lb # [ 88.731441] niks3-server[1044]: 2026/09/22 13:47:06 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4896worker1 # [ 88.669778] nix-grpc-daemon[765]: ts=2026-09-22T13:47:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_in=4324897worker1 # [ 88.687892] nix-grpc-daemon[765]: ts=2026-09-22T13:47:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784898worker1 # [ 88.700502] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4899worker1 # [ 88.704398] nix-daemon[833]: reaped child process 1030, status = succeeded4900client # [ 88.767948] nix[1023]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4901worker1 # [ 88.720245] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4902client # [ 89.112705] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4903client # Running as unit: share2.service; invocation ID: 407207e2636444c2b96d2577c6b171924904client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.10 seconds)4905worker2 # [ 89.282658] nix-grpc-daemon[774]: ts=2026-09-22T13:47:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443384906client # [ 89.331498] nix[1039]: don't know how to build these paths:4907client # [ 89.332775] nix[1039]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4908worker2 # [ 89.299352] nix-grpc-daemon[774]: ts=2026-09-22T13:47:06Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04909worker1 # [ 89.294149] nix-grpc-daemon[765]: ts=2026-09-22T13:47:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487784910client # [ 89.352666] nix[1039]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4911lb # [ 90.095129] envoy[913]: conn peer=[2001:db8:1::1]:55584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=951 ms=67644912client: must succeed: sleep 2; systemctl kill -s INT share14913lb # [ 90.100383] envoy[913]: conn peer=[2001:db8:1::1]:56340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=979 ms=68444914client # [ 92.487720] systemd[1]: share1.service: Sent signal SIGINT to main process 1023 (nix) on client request.4915client # [ 92.495754] nix[1023]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4916client # [ 92.509243] nix[1023]: error: interrupted by the user4917client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4918client: waiting for success: ! systemctl is-active share14919client # [ 92.566272] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4920client # [ 92.568814] systemd[1]: share1.service: Failed with result 'exit-code'.4921client # [ 92.571641] systemd[1]: share1.service: Consumed 178ms CPU time over 4.552s wall clock time, 39.4M memory peak, 6.3K incoming IP traffic, 8.2K outgoing IP traffic.4922client: (finished: waiting for success: ! systemctl is-active share1, in 1.11 seconds)4923client: waiting for success: ! systemctl is-active share24924worker1 # [ 93.987238] nix-daemon[833]: reaped child process 1044, status = succeeded4925worker1 # [ 94.122851] nix-daemon[833]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4926worker1 # [ 94.145535] nix-daemon[833]: reaped child process 1081, status = succeeded4927lb # [ 94.211687] niks3-server[1044]: 2026/09/22 13:47:11 INFO Received uploads request method=POST path=/api/pending_closures4928worker1 # [ 94.158065] nix-grpc-daemon[858]: time=2026-09-22T13:47:11.770Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4929worker1 # [ 94.162557] nix-grpc-daemon[858]: time=2026-09-22T13:47:11.772Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4930lb # [ 94.256454] niks3-server[1044]: 2026/09/22 13:47:11 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4931lb # [ 94.272491] niks3-server[1044]: 2026/09/22 13:47:11 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4932worker1 # [ 94.214935] nix-grpc-daemon[858]: time=2026-09-22T13:47:11.826Z level=INFO msg="Uploading 1 narinfos"4933lb # [ 94.283049] niks3-server[1044]: 2026/09/22 13:47:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4934lb # [ 94.289510] niks3-server[1044]: 2026/09/22 13:47:11 INFO Signed narinfos id=20 count=14935lb # [ 94.291429] niks3-server[1044]: 2026/09/22 13:47:11 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4936lb # [ 94.301478] niks3-server[1044]: 2026/09/22 13:47:11 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4937lb # [ 94.308464] niks3-server[1044]: 2026/09/22 13:47:11 INFO Completed upload id=204938worker1 # [ 94.241761] nix-grpc-daemon[858]: time=2026-09-22T13:47:11.853Z level=INFO msg="Upload complete. (302ms)"4939lb # [ 94.310265] niks3-server[1044]: 2026/09/22 13:47:11 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4940worker1 # [ 94.253253] nix-grpc-daemon[765]: ts=2026-09-22T13:47:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14941worker1 # [ 94.273742] nix-grpc-daemon[765]: ts=2026-09-22T13:47:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14942worker1 # [ 94.280858] nix-daemon[833]: reaped child process 1042, status = succeeded4943client # [ 94.337499] systemd[1]: share2.service: Deactivated successfully.4944client # [ 94.342506] systemd[1]: share2.service: Consumed 189ms CPU time over 5.226s wall clock time, 33.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4945client: (finished: waiting for success: ! systemctl is-active share2, in 1.13 seconds)4946client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4947client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4948(finished: subtest: the first client leaving does not fail the second, in 7.09 seconds)4949subtest: build hook: nix-daemon with builders = grpc://lb4950client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4951client # [ 95.175916] systemd[1]: Started Nix Daemon.4952client # [ 95.296034] nix-daemon[1091]: accepted connection from pid 1088, user root (trusted)4953worker2 # [ 95.403651] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:443384954worker2 # [ 95.410674] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443384955worker2 # [ 95.425138] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04956worker2 # [ 95.432541] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14957worker2 # [ 95.441841] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384958worker2 # [ 95.453165] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443384959worker2 # [ 95.468735] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443384960worker2 # [ 95.483954] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4961worker2 # [ 95.640269] nix-daemon[862]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)4962worker2 # [ 95.660676] nix-daemon[862]: reaped child process 1198, status = succeeded4963worker2 # [ 95.761997] nix-daemon[862]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)4964worker2 # [ 95.781260] nix-daemon[862]: reaped child process 1203, status = succeeded4965lb # [ 95.837423] niks3-server[1044]: 2026/09/22 13:47:13 INFO Received uploads request method=POST path=/api/pending_closures4966worker2 # [ 95.794620] nix-grpc-daemon[876]: time=2026-09-22T13:47:13.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4967worker2 # [ 95.796134] nix-grpc-daemon[876]: time=2026-09-22T13:47:13.392Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4968lb # [ 95.874743] niks3-server[1044]: 2026/09/22 13:47:13 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4969worker2 # [ 95.835488] nix-grpc-daemon[876]: time=2026-09-22T13:47:13.433Z level=INFO msg="Uploading 1 narinfos"4970lb # [ 95.890974] niks3-server[1044]: 2026/09/22 13:47:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4971lb # [ 95.902868] niks3-server[1044]: 2026/09/22 13:47:13 INFO Signed narinfos id=21 count=14972lb # [ 95.910978] niks3-server[1044]: 2026/09/22 13:47:13 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4973lb # [ 95.923875] niks3-server[1044]: 2026/09/22 13:47:13 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4974lb # [ 95.931599] niks3-server[1044]: 2026/09/22 13:47:13 INFO Completed upload id=214975lb # [ 95.934184] niks3-server[1044]: 2026/09/22 13:47:13 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4976worker2 # [ 95.874363] nix-grpc-daemon[876]: time=2026-09-22T13:47:13.472Z level=INFO msg="Upload complete. (374ms)"4977worker2 # [ 95.884598] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_in=5124978worker2 # [ 95.902695] nix-grpc-daemon[774]: ts=2026-09-22T13:47:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443384979worker2 # [ 95.910144] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4980worker2 # [ 95.915899] nix-daemon[862]: reaped child process 1193, status = succeeded4981worker2 # [ 95.929659] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4982worker2 # [ 96.991457] nix-daemon[862]: reaped child process 1207, status = succeeded4983worker2 # [ 97.115474] nix-daemon[862]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)4984worker2 # [ 97.135078] nix-daemon[862]: reaped child process 1226, status = succeeded4985lb # [ 97.198402] niks3-server[1044]: 2026/09/22 13:47:14 INFO Received uploads request method=POST path=/api/pending_closures4986worker2 # [ 97.156479] nix-grpc-daemon[876]: time=2026-09-22T13:47:14.752Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4987worker2 # [ 97.158245] nix-grpc-daemon[876]: time=2026-09-22T13:47:14.752Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4988lb # [ 97.245578] niks3-server[1044]: 2026/09/22 13:47:14 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4989lb # [ 97.269320] niks3-server[1044]: 2026/09/22 13:47:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4990worker2 # [ 97.222688] nix-grpc-daemon[876]: time=2026-09-22T13:47:14.820Z level=INFO msg="Uploading 1 narinfos"4991lb # [ 97.283706] niks3-server[1044]: 2026/09/22 13:47:14 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4992lb # [ 97.293619] niks3-server[1044]: 2026/09/22 13:47:14 INFO Signed narinfos id=22 count=14993lb # [ 97.299305] niks3-server[1044]: 2026/09/22 13:47:14 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4994worker2 # [ 97.252096] nix-grpc-daemon[876]: time=2026-09-22T13:47:14.850Z level=INFO msg="Upload complete. (308ms)"4995lb # [ 97.309399] niks3-server[1044]: 2026/09/22 13:47:14 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4996lb # [ 97.312962] niks3-server[1044]: 2026/09/22 13:47:14 INFO Completed upload id=224997lb # [ 97.316888] niks3-server[1044]: 2026/09/22 13:47:14 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4998worker2 # [ 97.265042] nix-grpc-daemon[774]: ts=2026-09-22T13:47:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14999worker2 # [ 97.285230] nix-grpc-daemon[774]: ts=2026-09-22T13:47:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443385000worker2 # [ 97.293173] nix-daemon[862]: reaped child process 1205, status = succeeded5001worker1 # [ 97.305998] nix-grpc-daemon[765]: ts=2026-09-22T13:47:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:487785002worker1 # [ 97.320616] nix-grpc-daemon[765]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...5003worker1 # [ 97.338295] nix-grpc-daemon[765]: ts=2026-09-22T13:47:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_out=1285004worker1 # [ 97.625574] nix-grpc-daemon[765]: ts=2026-09-22T13:47:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:487785005worker2 # [ 97.641028] nix-grpc-daemon[774]: ts=2026-09-22T13:47:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443385006worker2 # [ 97.652135] nix-grpc-daemon[876]: time=2026-09-22T13:47:15.250Z level=INFO msg="All 1 paths already cached"5007worker2 # [ 97.671940] nix-grpc-daemon[774]: ts=2026-09-22T13:47:15Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=05008worker2 # [ 97.678996] nix-grpc-daemon[774]: ts=2026-09-22T13:47:15Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=15009worker2 # [ 97.687488] nix-grpc-daemon[774]: ts=2026-09-22T13:47:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443385010worker2 # [ 97.698506] nix-grpc-daemon[774]: ts=2026-09-22T13:47:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443385011worker2 # [ 97.714214] nix-grpc-daemon[876]: time=2026-09-22T13:47:15.312Z level=INFO msg="All 1 paths already cached"5012worker2 # [ 97.724539] nix-grpc-daemon[774]: ts=2026-09-22T13:47:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443385013worker2 # [ 97.739311] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5014worker2 # [ 97.895612] nix-daemon[862]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5015worker2 # [ 97.913869] nix-daemon[862]: reaped child process 1235, status = succeeded5016worker2 # [ 98.016824] nix-daemon[862]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)5017worker2 # [ 98.034062] nix-daemon[862]: reaped child process 1240, status = succeeded5018lb # [ 98.094333] niks3-server[1044]: 2026/09/22 13:47:15 INFO Received uploads request method=POST path=/api/pending_closures5019worker2 # [ 98.051679] nix-grpc-daemon[876]: time=2026-09-22T13:47:15.649Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5020worker2 # [ 98.053286] nix-grpc-daemon[876]: time=2026-09-22T13:47:15.649Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"5021lb # [ 98.128708] niks3-server[1044]: 2026/09/22 13:47:15 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst5022lb # [ 98.146274] niks3-server[1044]: 2026/09/22 13:47:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5023worker2 # [ 98.096648] nix-grpc-daemon[876]: time=2026-09-22T13:47:15.694Z level=INFO msg="Uploading 1 narinfos"5024lb # [ 98.159342] niks3-server[1044]: 2026/09/22 13:47:15 INFO Signed narinfos id=23 count=15025lb # [ 98.166948] niks3-server[1044]: 2026/09/22 13:47:15 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls5026lb # [ 98.182771] niks3-server[1044]: 2026/09/22 13:47:15 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5027worker2 # [ 98.133762] nix-grpc-daemon[876]: time=2026-09-22T13:47:15.732Z level=INFO msg="Upload complete. (376ms)"5028lb # [ 98.191611] niks3-server[1044]: 2026/09/22 13:47:15 INFO Completed upload id=235029lb # [ 98.194511] niks3-server[1044]: 2026/09/22 13:47:15 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo5030worker2 # [ 98.144411] nix-grpc-daemon[774]: ts=2026-09-22T13:47:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_in=6245031worker2 # [ 98.160759] nix-grpc-daemon[774]: ts=2026-09-22T13:47:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443385032worker2 # [ 98.171269] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5033worker2 # [ 98.180260] nix-daemon[862]: reaped child process 1230, status = succeeded5034worker2 # [ 98.188269] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5035worker2 # [ 99.037491] nix-daemon[862]: reaped child process 1244, status = succeeded5036worker2 # [ 99.155070] nix-daemon[862]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5037worker2 # [ 99.177603] nix-daemon[862]: reaped child process 1263, status = succeeded5038lb # [ 99.233258] niks3-server[1044]: 2026/09/22 13:47:16 INFO Received uploads request method=POST path=/api/pending_closures5039worker2 # [ 99.187001] nix-grpc-daemon[876]: time=2026-09-22T13:47:16.785Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5040worker2 # [ 99.192172] nix-grpc-daemon[876]: time=2026-09-22T13:47:16.787Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"5041lb # [ 99.272858] niks3-server[1044]: 2026/09/22 13:47:16 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst5042lb # [ 99.287905] niks3-server[1044]: 2026/09/22 13:47:16 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv5043worker2 # [ 99.244250] nix-grpc-daemon[876]: time=2026-09-22T13:47:16.842Z level=INFO msg="Uploading 1 narinfos"5044lb # [ 99.299494] niks3-server[1044]: 2026/09/22 13:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5045lb # [ 99.308726] niks3-server[1044]: 2026/09/22 13:47:16 INFO Signed narinfos id=24 count=15046lb # [ 99.311071] niks3-server[1044]: 2026/09/22 13:47:16 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls5047lb # [ 99.325149] niks3-server[1044]: 2026/09/22 13:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5048lb # [ 99.331758] niks3-server[1044]: 2026/09/22 13:47:16 INFO Completed upload id=245049worker2 # [ 99.275681] nix-grpc-daemon[876]: time=2026-09-22T13:47:16.874Z level=INFO msg="Upload complete. (277ms)"5050lb # [ 99.335860] niks3-server[1044]: 2026/09/22 13:47:16 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo5051worker2 # [ 99.288703] nix-grpc-daemon[774]: ts=2026-09-22T13:47:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=15052worker2 # [ 99.311174] nix-grpc-daemon[774]: ts=2026-09-22T13:47:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443385053worker2 # [ 99.316479] nix-daemon[862]: reaped child process 1242, status = succeeded5054worker1 # [ 99.314628] nix-grpc-daemon[765]: ts=2026-09-22T13:47:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:487785055worker1 # [ 99.327495] nix-grpc-daemon[765]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...5056worker1 # [ 99.343519] nix-grpc-daemon[765]: ts=2026-09-22T13:47:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_out=1765057worker1 # [ 99.620823] nix-grpc-daemon[765]: ts=2026-09-22T13:47:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:487785058worker1 # [ 99.628430] nix-grpc-daemon[765]: ts=2026-09-22T13:47:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487785059worker1 # [ 99.636213] nix-grpc-daemon[858]: time=2026-09-22T13:47:17.246Z level=INFO msg="All 2 paths already cached"5060worker2 # [ 99.659777] nix-grpc-daemon[774]: ts=2026-09-22T13:47:17Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=05061worker2 # [ 99.668689] nix-grpc-daemon[774]: ts=2026-09-22T13:47:17Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=15062worker2 # [ 99.675360] nix-grpc-daemon[774]: ts=2026-09-22T13:47:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443385063worker2 # [ 99.686876] nix-grpc-daemon[774]: ts=2026-09-22T13:47:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443385064worker2 # [ 99.701042] nix-grpc-daemon[876]: time=2026-09-22T13:47:17.299Z level=INFO msg="All 2 paths already cached"5065worker2 # [ 99.711550] nix-grpc-daemon[774]: ts=2026-09-22T13:47:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443385066worker2 # [ 99.725870] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5067worker2 # [ 99.884157] nix-daemon[862]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5068worker2 # [ 99.908546] nix-daemon[862]: reaped child process 1272, status = succeeded5069worker2 # [ 100.005441] nix-daemon[862]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5070worker2 # [ 100.024573] nix-daemon[862]: reaped child process 1277, status = succeeded5071lb # [ 100.086514] niks3-server[1044]: 2026/09/22 13:47:17 INFO Received uploads request method=POST path=/api/pending_closures5072lb # [ 100.094948] envoy[913]: conn peer=[2001:db8:1::1]:56356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2990 tx=812 ms=42965073lb # [ 100.099984] envoy[913]: conn peer=[2001:db8:1::1]:56372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1456 tx=1194 ms=49955074worker2 # [ 100.052313] nix-grpc-daemon[876]: time=2026-09-22T13:47:17.650Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5075worker2 # [ 100.053905] nix-grpc-daemon[876]: time=2026-09-22T13:47:17.650Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"5076lb # [ 100.108114] envoy[913]: conn peer=[2001:db8:1::1]:55336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3418 tx=1968 ms=20075077lb # [ 100.114736] envoy[913]: conn peer=[2001:db8:1::1]:55340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=965078lb # [ 100.123967] envoy[913]: conn peer=[2001:db8:1::1]:55352 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2158 ms=17775079lb # [ 100.135169] envoy[913]: conn peer=[2001:db8:1::1]:55366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=915080lb # [ 100.147096] niks3-server[1044]: 2026/09/22 13:47:17 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst5081lb # [ 100.158409] niks3-server[1044]: 2026/09/22 13:47:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5082worker2 # [ 100.106573] nix-grpc-daemon[876]: time=2026-09-22T13:47:17.704Z level=INFO msg="Uploading 1 narinfos"5083lb # [ 100.166839] niks3-server[1044]: 2026/09/22 13:47:17 INFO Signed narinfos id=25 count=15084lb # [ 100.171251] niks3-server[1044]: 2026/09/22 13:47:17 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls5085lb # [ 100.178848] niks3-server[1044]: 2026/09/22 13:47:17 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5086lb # [ 100.186028] niks3-server[1044]: 2026/09/22 13:47:17 INFO Completed upload id=255087worker2 # [ 100.129496] nix-grpc-daemon[876]: time=2026-09-22T13:47:17.727Z level=INFO msg="Upload complete. (384ms)"5088lb # [ 100.188325] niks3-server[1044]: 2026/09/22 13:47:17 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo5089worker2 # [ 100.140645] nix-grpc-daemon[774]: ts=2026-09-22T13:47:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_in=7525090worker2 # [ 100.152995] nix-grpc-daemon[774]: ts=2026-09-22T13:47:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443385091worker2 # [ 100.166753] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5092worker2 # [ 100.171708] nix-daemon[862]: reaped child process 1267, status = succeeded5093worker2 # [ 100.186144] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5094worker2 # [ 101.616786] nix-daemon[862]: reaped child process 1281, status = succeeded5095worker2 # [ 101.719080] nix-daemon[862]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5096worker2 # [ 101.741197] nix-daemon[862]: reaped child process 1300, status = succeeded5097lb # [ 101.798523] niks3-server[1044]: 2026/09/22 13:47:19 INFO Received uploads request method=POST path=/api/pending_closures5098worker2 # [ 101.751683] nix-grpc-daemon[876]: time=2026-09-22T13:47:19.349Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5099worker2 # [ 101.753349] nix-grpc-daemon[876]: time=2026-09-22T13:47:19.349Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"5100lb # [ 101.833701] niks3-server[1044]: 2026/09/22 13:47:19 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv5101lb # [ 101.847196] niks3-server[1044]: 2026/09/22 13:47:19 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst5102lb # [ 101.860358] niks3-server[1044]: 2026/09/22 13:47:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5103worker2 # [ 101.810224] nix-grpc-daemon[876]: time=2026-09-22T13:47:19.408Z level=INFO msg="Uploading 1 narinfos"5104lb # [ 101.869722] niks3-server[1044]: 2026/09/22 13:47:19 INFO Signed narinfos id=26 count=15105lb # [ 101.875103] niks3-server[1044]: 2026/09/22 13:47:19 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls5106lb # [ 101.885432] niks3-server[1044]: 2026/09/22 13:47:19 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5107lb # [ 101.892337] niks3-server[1044]: 2026/09/22 13:47:19 INFO Completed upload id=265108worker2 # [ 101.836004] nix-grpc-daemon[876]: time=2026-09-22T13:47:19.434Z level=INFO msg="Upload complete. (239ms)"5109lb # [ 101.895253] niks3-server[1044]: 2026/09/22 13:47:19 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo5110worker2 # [ 101.848988] nix-grpc-daemon[774]: ts=2026-09-22T13:47:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=15111worker1 # [ 101.843928] nix-grpc-daemon[765]: ts=2026-09-22T13:47:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487785112worker1 # [ 101.852094] nix-grpc-daemon[765]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...5113worker2 # [ 101.867797] nix-daemon[862]: reaped child process 1279, status = succeeded5114worker2 # [ 101.898158] nix-grpc-daemon[774]: ts=2026-09-22T13:47:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443385115worker2 # [ 101.905154] nix-grpc-daemon[774]: ts=2026-09-22T13:47:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_out=2325116client # [ 102.035864] nix-daemon[1091]: reaped child process 1093, status = succeeded5117client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.02 seconds)5118client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top5119client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)5120client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5121client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)5122client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5123client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5124(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.14 seconds)5125subtest: an input only one worker has still reaches the builder5126client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only5127client # this derivation will be built:5128client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv5129client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...5130client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)5131client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure5132client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)5133worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5134worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)5135worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only5136worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)5137client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&25138client # [ 103.006371] nix-daemon[1091]: accepted connection from pid 1173, user root (trusted)5139client # this derivation will be built:5140client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5141client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5142worker2 # [ 103.116899] nix-grpc-daemon[774]: ts=2026-09-22T13:47:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:443385143worker2 # [ 103.124702] nix-grpc-daemon[774]: ts=2026-09-22T13:47:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443385144client # copying 1 paths...5145client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...5146worker2 # [ 103.135725] nix-grpc-daemon[774]: ts=2026-09-22T13:47:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443385147worker2 # [ 103.151165] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5148worker2 # [ 103.318034] nix-daemon[862]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5149worker2 # [ 103.337254] nix-daemon[862]: reaped child process 1310, status = succeeded5150lb # [ 103.397906] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received uploads request method=POST path=/api/pending_closures5151worker2 # [ 103.351828] nix-grpc-daemon[876]: time=2026-09-22T13:47:20.949Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5152worker2 # [ 103.353557] nix-grpc-daemon[876]: time=2026-09-22T13:47:20.949Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"5153lb # [ 103.432497] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5154lb # [ 103.447140] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5155worker2 # [ 103.396920] nix-grpc-daemon[876]: time=2026-09-22T13:47:20.995Z level=INFO msg="Uploading 1 narinfos"5156lb # [ 103.457815] niks3-server[1044]: 2026/09/22 13:47:21 INFO Signed narinfos id=27 count=15157lb # [ 103.463810] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls5158lb # [ 103.472198] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5159lb # [ 103.481190] niks3-server[1044]: 2026/09/22 13:47:21 INFO Completed upload id=275160worker2 # [ 103.425039] nix-grpc-daemon[876]: time=2026-09-22T13:47:21.023Z level=INFO msg="Upload complete. (257ms)"5161lb # [ 103.484636] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo5162worker2 # [ 103.436417] nix-grpc-daemon[774]: ts=2026-09-22T13:47:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_in=1205163worker2 # [ 103.458519] nix-grpc-daemon[774]: ts=2026-09-22T13:47:21Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=05164worker2 # [ 103.463627] nix-grpc-daemon[774]: ts=2026-09-22T13:47:21Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15165worker2 # [ 103.468926] nix-daemon[862]: reaped child process 1305, status = succeeded5166worker1 # [ 103.487349] nix-grpc-daemon[765]: ts=2026-09-22T13:47:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487785167worker1 # [ 103.503299] nix-grpc-daemon[765]: ts=2026-09-22T13:47:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487785168client # copying 2 paths to 'lb:50051'...5169worker1 # [ 103.520084] nix-grpc-daemon[765]: ts=2026-09-22T13:47:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487785170client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...5171client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...5172worker1 # [ 103.536162] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5173worker1 # [ 103.699515] nix-daemon[833]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5174worker1 # [ 103.717908] nix-daemon[833]: reaped child process 1096, status = succeeded5175worker1 # [ 103.820422] nix-daemon[833]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)5176worker1 # [ 103.837748] nix-daemon[833]: reaped child process 1101, status = succeeded5177lb # [ 103.909100] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received uploads request method=POST path=/api/pending_closures5178lb # [ 103.918678] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received uploads request method=POST path=/api/pending_closures5179worker1 # [ 103.863066] nix-grpc-daemon[858]: time=2026-09-22T13:47:21.474Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5180worker1 # [ 103.864630] nix-grpc-daemon[858]: time=2026-09-22T13:47:21.474Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"5181worker1 # [ 103.866362] nix-grpc-daemon[858]: time=2026-09-22T13:47:21.474Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"5182lb # [ 103.959262] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst5183lb # [ 103.970496] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst5184lb # [ 103.983810] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5185worker1 # [ 103.924087] nix-grpc-daemon[858]: time=2026-09-22T13:47:21.535Z level=INFO msg="Uploading 3 narinfos"5186lb # [ 103.994025] niks3-server[1044]: 2026/09/22 13:47:21 INFO Signed narinfos id=28 count=15187lb # [ 104.000241] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5188lb # [ 104.003398] niks3-server[1044]: 2026/09/22 13:47:21 INFO Signed narinfos id=29 count=25189lb # [ 104.011521] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5190lb # [ 104.017257] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls5191lb # [ 104.031984] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5192lb # [ 104.039891] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5193lb # [ 104.047323] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5194lb # [ 104.054194] niks3-server[1044]: 2026/09/22 13:47:21 INFO Completed upload id=295195lb # [ 104.056896] niks3-server[1044]: 2026/09/22 13:47:21 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5196worker1 # [ 103.990354] nix-grpc-daemon[858]: time=2026-09-22T13:47:21.602Z level=INFO msg="Upload complete. (434ms)"5197lb # [ 104.061897] niks3-server[1044]: 2026/09/22 13:47:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5198lb # [ 104.066017] niks3-server[1044]: 2026/09/22 13:47:21 INFO Completed upload id=285199worker1 # [ 104.003361] nix-grpc-daemon[765]: ts=2026-09-22T13:47:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=2 nar_bytes_in=9845200worker1 # [ 104.020586] nix-grpc-daemon[765]: ts=2026-09-22T13:47:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487785201worker1 # [ 104.029457] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5202worker1 # [ 104.034942] nix-daemon[833]: reaped child process 1091, status = succeeded5203client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5204worker1 # [ 104.047245] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5205client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5206client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5207worker1 # [ 105.096703] nix-daemon[833]: reaped child process 1105, status = succeeded5208worker1 # [ 105.173449] nix-daemon[833]: accepted connection from pid 1121, user nix-grpc-daemon (trusted)5209worker1 # [ 105.190346] nix-daemon[833]: reaped child process 1124, status = succeeded5210lb # [ 105.265404] niks3-server[1044]: 2026/09/22 13:47:22 INFO Received uploads request method=POST path=/api/pending_closures5211worker1 # [ 105.212421] nix-grpc-daemon[858]: time=2026-09-22T13:47:22.823Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5212lb # [ 105.299334] niks3-server[1044]: 2026/09/22 13:47:22 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5213worker1 # [ 105.244461] nix-grpc-daemon[858]: time=2026-09-22T13:47:22.856Z level=INFO msg="Uploading 1 narinfos"5214lb # [ 105.314034] niks3-server[1044]: 2026/09/22 13:47:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5215lb # [ 105.324067] niks3-server[1044]: 2026/09/22 13:47:22 INFO Signed narinfos id=30 count=15216lb # [ 105.329966] niks3-server[1044]: 2026/09/22 13:47:22 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5217worker1 # [ 105.273546] nix-grpc-daemon[858]: time=2026-09-22T13:47:22.885Z level=INFO msg="Upload complete. (214ms)"5218lb # [ 105.340671] niks3-server[1044]: 2026/09/22 13:47:22 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5219lb # [ 105.349913] niks3-server[1044]: 2026/09/22 13:47:22 INFO Completed upload id=305220worker1 # [ 105.282041] nix-grpc-daemon[765]: ts=2026-09-22T13:47:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15221client # copying 1 paths...5222lb # [ 105.353113] niks3-server[1044]: 2026/09/22 13:47:22 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5223client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5224worker1 # [ 105.300764] nix-grpc-daemon[765]: ts=2026-09-22T13:47:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487785225worker1 # [ 105.304397] nix-daemon[833]: reaped child process 1103, status = succeeded5226worker1 # [ 105.319146] nix-grpc-daemon[765]: ts=2026-09-22T13:47:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:487785227worker1 # [ 105.325217] nix-grpc-daemon[765]: ts=2026-09-22T13:47:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_out=1205228client # [ 105.506237] nix-daemon[1091]: reaped child process 1176, status = succeeded5229client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.69 seconds)5230client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25231client # [ 105.690601] nix-daemon[1091]: accepted connection from pid 1195, user root (trusted)5232client # this derivation will be built:5233client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5234worker1 # [ 105.784579] nix-grpc-daemon[765]: ts=2026-09-22T13:47:23Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:487785235client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5236worker2 # [ 105.801061] nix-grpc-daemon[774]: ts=2026-09-22T13:47:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443385237worker2 # [ 105.812002] nix-grpc-daemon[876]: time=2026-09-22T13:47:23.409Z level=INFO msg="All 1 paths already cached"5238worker2 # [ 105.825086] nix-grpc-daemon[774]: ts=2026-09-22T13:47:23Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05239worker2 # [ 105.832318] nix-grpc-daemon[774]: ts=2026-09-22T13:47:23Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15240worker2 # [ 105.841138] nix-grpc-daemon[774]: ts=2026-09-22T13:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443385241worker2 # [ 105.852286] nix-grpc-daemon[774]: ts=2026-09-22T13:47:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443385242client # copying 1 paths to 'lb:50051'...5243client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5244worker2 # [ 105.873310] nix-grpc-daemon[774]: ts=2026-09-22T13:47:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443385245worker2 # [ 105.887798] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5246worker2 # [ 105.908490] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)5247worker2 # [ 105.915266] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 2/5)5248worker2 # [ 105.934173] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 228 ms (attempt 3/5)5249worker2 # [ 106.164385] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 542 ms (attempt 4/5)5250worker2 # [ 106.708976] nix-grpc-daemon[774]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5251worker2 # [ 106.717361] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5252worker2 # [ 106.724205] nix-grpc-daemon[774]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5253worker2 # [ 106.901992] nix-daemon[862]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5254worker2 # [ 106.923688] nix-daemon[862]: reaped child process 1326, status = succeeded5255worker2 # [ 107.031635] nix-daemon[862]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5256worker2 # [ 107.049720] nix-daemon[862]: reaped child process 1331, status = succeeded5257lb # [ 107.115090] niks3-server[1044]: 2026/09/22 13:47:24 INFO Received uploads request method=POST path=/api/pending_closures5258worker2 # [ 107.067707] nix-grpc-daemon[876]: time=2026-09-22T13:47:24.665Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5259worker2 # [ 107.069461] nix-grpc-daemon[876]: time=2026-09-22T13:47:24.665Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5260lb # [ 107.155150] niks3-server[1044]: 2026/09/22 13:47:24 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5261worker2 # [ 107.114165] nix-grpc-daemon[876]: time=2026-09-22T13:47:24.712Z level=INFO msg="Uploading 1 narinfos"5262lb # [ 107.172224] niks3-server[1044]: 2026/09/22 13:47:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5263lb # [ 107.175716] niks3-server[1044]: 2026/09/22 13:47:24 INFO Signed narinfos id=31 count=15264lb # [ 107.186991] niks3-server[1044]: 2026/09/22 13:47:24 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5265lb # [ 107.195342] niks3-server[1044]: 2026/09/22 13:47:24 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5266lb # [ 107.200758] niks3-server[1044]: 2026/09/22 13:47:24 INFO Completed upload id=315267worker2 # [ 107.144583] nix-grpc-daemon[876]: time=2026-09-22T13:47:24.743Z level=INFO msg="Upload complete. (398ms)"5268lb # [ 107.207399] niks3-server[1044]: 2026/09/22 13:47:24 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5269worker2 # [ 107.157342] nix-grpc-daemon[774]: ts=2026-09-22T13:47:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=1 paths=1 nar_bytes_in=5685270worker2 # [ 107.168938] nix-grpc-daemon[774]: ts=2026-09-22T13:47:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443385271worker2 # [ 107.182326] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5272worker2 # [ 107.187904] nix-daemon[862]: reaped child process 1316, status = succeeded5273worker2 # [ 107.200296] nix-daemon[862]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5274client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5275client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5276client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5277worker2 # [ 108.086999] nix-daemon[862]: reaped child process 1335, status = succeeded5278worker2 # [ 108.156243] nix-daemon[862]: accepted connection from pid 1351, user nix-grpc-daemon (trusted)5279worker2 # [ 108.175864] nix-daemon[862]: reaped child process 1354, status = succeeded5280lb # [ 108.242188] niks3-server[1044]: 2026/09/22 13:47:25 INFO Received uploads request method=POST path=/api/pending_closures5281worker2 # [ 108.196420] nix-grpc-daemon[876]: time=2026-09-22T13:47:25.792Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5282worker2 # [ 108.226538] nix-grpc-daemon[876]: time=2026-09-22T13:47:25.824Z level=INFO msg="Uploading 1 narinfos"5283lb # [ 108.272376] niks3-server[1044]: 2026/09/22 13:47:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5284lb # [ 108.289095] niks3-server[1044]: 2026/09/22 13:47:25 INFO Signed narinfos id=32 count=15285lb # [ 108.296189] niks3-server[1044]: 2026/09/22 13:47:25 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5286lb # [ 108.307172] niks3-server[1044]: 2026/09/22 13:47:25 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5287worker2 # [ 108.258302] nix-grpc-daemon[876]: time=2026-09-22T13:47:25.856Z level=INFO msg="Upload complete. (244ms)"5288lb # [ 108.315154] niks3-server[1044]: 2026/09/22 13:47:25 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5289lb # [ 108.320789] niks3-server[1044]: 2026/09/22 13:47:25 INFO Completed upload id=325290lb # [ 108.323977] niks3-server[1044]: 2026/09/22 13:47:25 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5291worker2 # [ 108.269048] nix-grpc-daemon[774]: ts=2026-09-22T13:47:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15292worker2 # [ 108.284914] nix-daemon[862]: reaped child process 1333, status = succeeded5293client # copying 1 paths...5294worker2 # [ 108.314063] nix-grpc-daemon[774]: ts=2026-09-22T13:47:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443385295client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5296worker2 # [ 108.347913] nix-grpc-daemon[774]: ts=2026-09-22T13:47:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443385297worker2 # [ 108.354875] nix-grpc-daemon[774]: ts=2026-09-22T13:47:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44338 duration_s=0 paths=1 nar_bytes_out=1205298client # [ 108.521073] nix-daemon[1091]: reaped child process 1198, status = succeeded5299client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.01 seconds)5300client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25301client # [ 108.692402] nix-daemon[1091]: accepted connection from pid 1217, user root (trusted)5302client # this derivation will be built:5303client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5304client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5305worker1 # [ 108.786635] nix-grpc-daemon[765]: ts=2026-09-22T13:47:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:487785306worker1 # [ 108.794409] nix-grpc-daemon[765]: ts=2026-09-22T13:47:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487785307worker1 # [ 108.802253] nix-grpc-daemon[858]: time=2026-09-22T13:47:26.412Z level=INFO msg="All 1 paths already cached"5308worker2 # [ 108.824165] nix-grpc-daemon[774]: ts=2026-09-22T13:47:26Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05309worker2 # [ 108.830532] nix-grpc-daemon[774]: ts=2026-09-22T13:47:26Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15310worker1 # [ 108.823360] nix-grpc-daemon[765]: ts=2026-09-22T13:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487785311worker1 # [ 108.839353] nix-grpc-daemon[765]: ts=2026-09-22T13:47:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487785312worker1 # [ 108.854651] nix-grpc-daemon[858]: time=2026-09-22T13:47:26.466Z level=INFO msg="All 1 paths already cached"5313client # copying 1 paths to 'lb:50051'...5314client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5315worker1 # [ 108.865134] nix-grpc-daemon[765]: ts=2026-09-22T13:47:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487785316worker1 # [ 108.880899] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5317worker1 # [ 109.034685] nix-daemon[833]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)5318worker1 # [ 109.054463] nix-daemon[833]: reaped child process 1131, status = succeeded5319worker1 # [ 109.157056] nix-daemon[833]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5320worker1 # [ 109.175288] nix-daemon[833]: reaped child process 1136, status = succeeded5321lb # [ 109.246735] niks3-server[1044]: 2026/09/22 13:47:26 INFO Received uploads request method=POST path=/api/pending_closures5322worker1 # [ 109.190325] nix-grpc-daemon[858]: time=2026-09-22T13:47:26.802Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5323worker1 # [ 109.191916] nix-grpc-daemon[858]: time=2026-09-22T13:47:26.802Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5324lb # [ 109.275385] niks3-server[1044]: 2026/09/22 13:47:26 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5325lb # [ 109.294872] niks3-server[1044]: 2026/09/22 13:47:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5326worker1 # [ 109.234357] nix-grpc-daemon[858]: time=2026-09-22T13:47:26.846Z level=INFO msg="Uploading 1 narinfos"5327lb # [ 109.306489] niks3-server[1044]: 2026/09/22 13:47:26 INFO Signed narinfos id=33 count=15328lb # [ 109.314045] niks3-server[1044]: 2026/09/22 13:47:26 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5329worker1 # [ 109.260127] nix-grpc-daemon[858]: time=2026-09-22T13:47:26.872Z level=INFO msg="Upload complete. (362ms)"5330lb # [ 109.326792] niks3-server[1044]: 2026/09/22 13:47:26 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5331worker1 # [ 109.268151] nix-grpc-daemon[765]: ts=2026-09-22T13:47:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_in=5685332lb # [ 109.339146] niks3-server[1044]: 2026/09/22 13:47:26 INFO Completed upload id=335333lb # [ 109.343389] niks3-server[1044]: 2026/09/22 13:47:26 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5334worker1 # [ 109.284271] nix-grpc-daemon[765]: ts=2026-09-22T13:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487785335worker1 # [ 109.290667] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5336worker1 # [ 109.295626] nix-daemon[833]: reaped child process 1126, status = succeeded5337worker1 # [ 109.307723] nix-daemon[833]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5338client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5339client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5340client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5341lb # [ 110.096267] envoy[913]: conn peer=[2001:db8:1::1]:45970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=335342lb # [ 110.096636] envoy[913]: conn peer=[2001:db8:1::1]:55368 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2388 ms=22975343lb # [ 110.096760] envoy[913]: conn peer=[2001:db8:1::1]:45972 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4247 tx=2057 ms=22775344lb # [ 110.096893] envoy[913]: conn peer=[2001:db8:1::1]:45974 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=665345lb # [ 110.097026] envoy[913]: conn peer=[2001:db8:1::1]:45990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3743 tx=2123 ms=26175346lb # [ 110.097139] envoy[913]: conn peer=[2001:db8:1::1]:46004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=695347worker1 # [ 110.212914] nix-daemon[833]: reaped child process 1140, status = succeeded5348worker1 # [ 110.284461] nix-daemon[833]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)5349worker1 # [ 110.303658] nix-daemon[833]: reaped child process 1159, status = succeeded5350lb # [ 110.375028] niks3-server[1044]: 2026/09/22 13:47:27 INFO Received uploads request method=POST path=/api/pending_closures5351worker1 # [ 110.318693] nix-grpc-daemon[858]: time=2026-09-22T13:47:27.929Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5352lb # [ 110.407663] niks3-server[1044]: 2026/09/22 13:47:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5353worker1 # [ 110.349689] nix-grpc-daemon[858]: time=2026-09-22T13:47:27.961Z level=INFO msg="Uploading 1 narinfos"5354lb # [ 110.422959] niks3-server[1044]: 2026/09/22 13:47:28 INFO Signed narinfos id=34 count=15355lb # [ 110.431569] niks3-server[1044]: 2026/09/22 13:47:28 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5356lb # [ 110.444782] niks3-server[1044]: 2026/09/22 13:47:28 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5357worker1 # [ 110.386696] nix-grpc-daemon[858]: time=2026-09-22T13:47:27.999Z level=INFO msg="Upload complete. (266ms)"5358lb # [ 110.457169] niks3-server[1044]: 2026/09/22 13:47:28 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5359lb # [ 110.462504] niks3-server[1044]: 2026/09/22 13:47:28 INFO Completed upload id=345360lb # [ 110.465588] niks3-server[1044]: 2026/09/22 13:47:28 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5361client # copying 1 paths...5362worker1 # [ 110.401205] nix-grpc-daemon[765]: ts=2026-09-22T13:47:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15363worker1 # [ 110.417269] nix-grpc-daemon[765]: ts=2026-09-22T13:47:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487785364worker1 # [ 110.424445] nix-daemon[833]: reaped child process 1138, status = succeeded5365client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5366worker1 # [ 110.450733] nix-grpc-daemon[765]: ts=2026-09-22T13:47:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:487785367worker1 # [ 110.457699] nix-grpc-daemon[765]: ts=2026-09-22T13:47:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48778 duration_s=0 paths=1 nar_bytes_out=1205368client # [ 110.629775] nix-daemon[1091]: reaped child process 1220, status = succeeded5369client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.11 seconds)5370(finished: subtest: an input only one worker has still reaches the builder, in 8.46 seconds)5371subtest: upload whose reference the worker lacks is completed from the cache5372client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5373client # this derivation will be built:5374client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5375client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5376client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.32 seconds)5377client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5378client # this derivation will be built:5379client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5380client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5381client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.29 seconds)5382client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5383client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5384worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5385worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)5386worker2 # [ 111.589182] systemd[1]: Stopping Nix worker-protocol over gRPC...5387worker2 # [ 112.411328] nix-grpc-daemon[774]: ts=2026-09-22T13:47:30Z level=info event=signal_exit5388lb # [ 112.789466] niks3-server[1044]: 2026/09/22 13:47:30 INFO lead: released remote=192.168.1.4:484585389worker1 # [ 112.725698] nix-grpc-daemon[765]: ts=2026-09-22T13:47:30Z level=info event=scheduler_take_over5390lb # [ 112.794854] niks3-server[1044]: 2026/09/22 13:47:30 INFO lead: acquired remote=192.168.1.3:592365391worker2 # [ 113.730956] nix-grpc-daemon[774]: ts=2026-09-22T13:47:31Z level=info event=scheduler_restarting5392worker1 # [ 113.720980] nix-grpc-daemon[765]: ts=2026-09-22T13:47:31Z level=info event=scheduler_restarting addr=lb:500515393worker2 # [ 113.739460] nix-grpc-daemon[774]: ts=2026-09-22T13:47:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443305394worker2 # [ 113.749432] nix-grpc-daemon[774]: ts=2026-09-22T13:47:31Z level=info event=worker_gone addr=192.168.1.3:500515395worker2 # [ 113.755582] nix-grpc-daemon[774]: ts=2026-09-22T13:47:31Z level=info event=scheduler_restarting addr=lb:500515396worker2 # [ 113.760802] nix-grpc-daemon[774]: ts=2026-09-22T13:47:31Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443305397worker2 # [ 113.765656] nix-grpc-daemon[774]: ts=2026-09-22T13:47:31Z level=info event=worker_gone addr=192.168.1.4:500515398worker2 # [ 113.789202] nix-daemon[862]: reaped child process 864, status = succeeded5399worker1 # [ 113.924251] nix-grpc-daemon[765]: ts=2026-09-22T13:47:31Z level=info event=scheduler_connected addr=lb:500515400worker1 # [ 113.928457] nix-grpc-daemon[765]: ts=2026-09-22T13:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5401worker2 # [ 113.967086] nix-grpc-daemon[774]: ts=2026-09-22T13:47:31Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5402worker1 # [ 114.125096] nix-grpc-daemon[765]: ts=2026-09-22T13:47:31Z level=info event=scheduler_connected addr=lb:500515403worker1 # [ 114.130274] nix-grpc-daemon[765]: ts=2026-09-22T13:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5404worker1 # [ 114.331014] nix-grpc-daemon[765]: ts=2026-09-22T13:47:31Z level=info event=scheduler_connected addr=lb:500515405worker1 # [ 114.336244] nix-grpc-daemon[765]: ts=2026-09-22T13:47:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5406worker1 # [ 114.536818] nix-grpc-daemon[765]: ts=2026-09-22T13:47:32Z level=info event=scheduler_connected addr=lb:500515407worker2 # [ 114.562718] nix-grpc-daemon[774]: E0000 00:00:1790084852.160950 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5408worker1 # [ 114.556630] nix-grpc-daemon[765]: ts=2026-09-22T13:47:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:443205409worker1 # [ 114.562032] nix-grpc-daemon[765]: ts=2026-09-22T13:47:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05410worker2 # [ 114.584853] nix-grpc-daemon[774]: E0000 00:00:1790084852.183217 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5411worker2 # [ 115.737932] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5412worker2 # [ 115.742722] systemd[1]: Stopped Nix worker-protocol over gRPC.5413worker2 # [ 115.748881] systemd[1]: nix-grpc-daemon.service: Consumed 4.989s CPU time over 1min 35.574s wall clock time, 70.4M memory peak, 121K incoming IP traffic, 316K outgoing IP traffic.5414worker2 # [ 115.765670] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5415lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5416lb: must succeed: curl -sf localhost:9901/clusters5417worker2 # [ 115.779061] systemd[1]: Closed Nix worker-protocol over gRPC.5418worker2 # [ 115.781346] systemd[1]: nix-grpc-daemon.socket: Consumed 734us CPU time over 1min 45.452s wall clock time, 256K memory peak, 130.2K incoming IP traffic, 107.3K outgoing IP traffic.5419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5420lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5421client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25422worker1 # [ 116.087335] nix-grpc-daemon[765]: ts=2026-09-22T13:47:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487785423worker1 # [ 116.098956] nix-grpc-daemon[765]: ts=2026-09-22T13:47:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487785424client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5425client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.28 seconds)5426worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5427worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5428worker2 # [ 116.241792] systemd[1]: Listening on Nix worker-protocol over gRPC.5429worker1 # [ 116.307418] systemd[1]: Stopping Nix worker-protocol over gRPC...5430worker2 # [ 116.732700] systemd[1]: Starting scheduler-token.service...5431worker2 # [ 116.818663] systemd[1]: scheduler-token.service: Deactivated successfully.5432worker2 # [ 116.824695] systemd[1]: Finished scheduler-token.service.5433worker2 # [ 116.826255] systemd[1]: scheduler-token.service: Consumed 57ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5434worker2 # [ 116.834423] systemd[1]: Starting Nix worker-protocol over gRPC...5435worker2 # [ 116.968514] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5436worker2 # [ 116.970850] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5437worker2 # [ 116.974787] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15438worker2 # [ 116.980105] nix-grpc-daemon[1380]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5439worker2 # [ 116.995343] nix-grpc-daemon[1380]: E0000 00:00:1790084854.578451 1380 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5440worker2 # [ 117.001250] nix-grpc-daemon[1380]: E0000 00:00:1790084854.582267 1380 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5441worker2 # [ 117.032196] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5442worker2 # [ 117.033874] nix-grpc-daemon[1380]: E0000 00:00:1790084854.597186 1422 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5443worker2 # [ 117.040378] systemd[1]: Started Nix worker-protocol over gRPC.5444worker2 # [ 117.041776] nix-grpc-daemon[1380]: E0000 00:00:1790084854.618513 1422 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5445worker2 # [ 117.044290] nix-grpc-daemon[1380]: E0000 00:00:1790084854.618710 1422 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5446worker2 # [ 117.046574] nix-grpc-daemon[1380]: E0000 00:00:1790084854.637817 1422 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5447worker2 # [ 117.054968] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:34Z level=info event=scheduler_connected addr=lb:500515448worker1 # [ 117.056198] nix-grpc-daemon[765]: ts=2026-09-22T13:47:34Z level=info event=oidc_jwks provider=mock keys=15449worker1 # [ 117.061355] nix-grpc-daemon[765]: ts=2026-09-22T13:47:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:443205450worker1 # [ 117.066899] nix-grpc-daemon[765]: ts=2026-09-22T13:47:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05451worker1 # [ 117.303143] nix-grpc-daemon[765]: ts=2026-09-22T13:47:34Z level=info event=signal_exit5452lb # [ 117.789520] niks3-server[1044]: 2026/09/22 13:47:35 INFO lead: released remote=192.168.1.3:592365453worker1 # [ 118.721364] nix-grpc-daemon[765]: ts=2026-09-22T13:47:36Z level=info event=scheduler_restarting5454worker2 # [ 118.734607] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:36Z level=info event=scheduler_restarting addr=lb:500515455worker1 # [ 118.733656] nix-grpc-daemon[765]: ts=2026-09-22T13:47:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443205456worker1 # [ 118.738870] nix-grpc-daemon[765]: ts=2026-09-22T13:47:36Z level=info event=worker_gone addr=192.168.1.4:500515457worker1 # [ 118.743743] nix-grpc-daemon[765]: ts=2026-09-22T13:47:36Z level=info event=scheduler_restarting addr=lb:500515458worker1 # [ 118.748385] nix-grpc-daemon[765]: ts=2026-09-22T13:47:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443205459worker1 # [ 118.754817] nix-grpc-daemon[765]: ts=2026-09-22T13:47:36Z level=info event=worker_gone addr=192.168.1.3:500515460worker1 # [ 118.778942] nix-daemon[833]: reaped child process 835, status = succeeded5461worker2 # [ 118.934574] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:36Z level=info event=scheduler_connected addr=lb:500515462worker2 # [ 118.940404] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5463worker1 # [ 118.951420] nix-grpc-daemon[765]: ts=2026-09-22T13:47:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5464worker2 # [ 119.141091] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:36Z level=info event=scheduler_connected addr=lb:500515465worker2 # [ 119.146372] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5466worker2 # [ 119.347367] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:36Z level=info event=scheduler_connected addr=lb:500515467worker2 # [ 119.352737] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5468worker1 # [ 119.531193] nix-grpc-daemon[765]: E0000 00:00:1790084857.143238 810 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5469worker1 # [ 119.538952] nix-grpc-daemon[765]: E0000 00:00:1790084857.150973 810 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5470worker2 # [ 119.553672] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_connected addr=lb:500515471worker2 # [ 119.558807] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5472worker1 # [ 119.660713] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5473worker1 # [ 119.667594] systemd[1]: Stopped Nix worker-protocol over gRPC.5474worker1 # [ 119.675482] systemd[1]: nix-grpc-daemon.service: Consumed 3.433s CPU time over 1min 39.631s wall clock time, 58.8M memory peak, 90.1K incoming IP traffic, 161.7K outgoing IP traffic.5475worker1 # [ 119.693242] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5476worker1 # [ 119.695436] systemd[1]: Closed Nix worker-protocol over gRPC.5477lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5478lb: must succeed: curl -sf localhost:9901/clusters5479worker1 # [ 119.702293] systemd[1]: nix-grpc-daemon.socket: Consumed 619us CPU time over 1min 49.255s wall clock time, 256K memory peak, 102.3K incoming IP traffic, 90.2K outgoing IP traffic.5480worker2 # [ 119.761091] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_connected addr=lb:500515481worker2 # [ 119.766089] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5483lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5484client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25485worker2 # [ 119.967110] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_connected addr=lb:500515486worker2 # [ 119.972243] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5487worker2 # [ 120.010484] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:350285488worker2 # [ 120.026043] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5489lb # [ 120.095101] envoy[913]: conn peer=[2001:db8:1::1]:46012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3744 tx=2100 ms=17285490lb # [ 120.100072] envoy[913]: conn peer=[2001:db8:1::1]:46022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=685491lb # [ 120.104604] envoy[913]: conn peer=[2001:db8:1::4]:33060 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=88247 tx=4747 ms=935245492worker2 # [ 120.051978] nix-grpc-daemon[1380]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5)5493lb # [ 120.109441] envoy[913]: conn peer=[2001:db8:1::1]:40036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=375494lb # [ 120.113888] envoy[913]: conn peer=[2001:db8:1::3]:43010 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=17683 tx=4716 ms=986125495worker2 # [ 120.101530] nix-grpc-daemon[1380]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 2/5)5496worker2 # [ 120.169377] nix-grpc-daemon[1380]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 3/5)5497worker2 # [ 120.177800] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_connected addr=lb:500515498worker2 # [ 120.182587] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker2 # [ 120.197406] nix-grpc-daemon[1380]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 400 ms (attempt 4/5)5500worker2 # [ 120.383872] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_connected addr=lb:500515501worker2 # [ 120.389061] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5502worker2 # [ 120.589925] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=scheduler_connected addr=lb:500515503worker2 # [ 120.594731] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5504worker2 # [ 120.606623] nix-grpc-daemon[1380]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5505worker2 # [ 120.612994] nix-grpc-daemon[1380]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5506client # copying 2 paths...5507client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5508worker2 # [ 120.635342] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:350285509client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5510worker2 # [ 120.650109] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5511worker2 # [ 120.683710] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=niks3_push_started pid=14405512worker2 # [ 120.796942] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=scheduler_connected addr=lb:500515513worker2 # [ 120.820188] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5514worker2 # [ 120.841560] nix-grpc-daemon[1440]: time=2026-09-22T13:47:38.439Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5515worker2 # [ 120.954331] nix-daemon[862]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)5516worker2 # [ 120.972756] nix-daemon[862]: reaped child process 1456, status = succeeded5517lb # [ 121.038105] niks3-server[1044]: 2026/09/22 13:47:38 INFO Received uploads request method=POST path=/api/pending_closures5518lb # [ 121.046096] niks3-server[1044]: 2026/09/22 13:47:38 INFO Received uploads request method=POST path=/api/pending_closures5519worker2 # [ 120.999733] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=scheduler_connected addr=lb:500515520worker2 # [ 121.004987] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5521worker2 # [ 121.006630] nix-grpc-daemon[1440]: time=2026-09-22T13:47:38.600Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5522worker2 # [ 121.008068] nix-grpc-daemon[1440]: time=2026-09-22T13:47:38.600Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5523worker2 # [ 121.009704] nix-grpc-daemon[1440]: time=2026-09-22T13:47:38.600Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5524lb # [ 121.103629] niks3-server[1044]: 2026/09/22 13:47:38 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5525lb # [ 121.119415] niks3-server[1044]: 2026/09/22 13:47:38 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5526lb # [ 121.129231] niks3-server[1044]: 2026/09/22 13:47:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5527worker2 # [ 121.081655] nix-grpc-daemon[1440]: time=2026-09-22T13:47:38.678Z level=INFO msg="Uploading 3 narinfos"5528lb # [ 121.138644] niks3-server[1044]: 2026/09/22 13:47:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5529lb # [ 121.146469] niks3-server[1044]: 2026/09/22 13:47:38 INFO Signed narinfos id=35 count=15530lb # [ 121.150999] niks3-server[1044]: 2026/09/22 13:47:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5531lb # [ 121.158893] niks3-server[1044]: 2026/09/22 13:47:38 INFO Signed narinfos id=36 count=25532lb # [ 121.162664] niks3-server[1044]: 2026/09/22 13:47:38 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5533lb # [ 121.179658] niks3-server[1044]: 2026/09/22 13:47:38 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5534lb # [ 121.190527] niks3-server[1044]: 2026/09/22 13:47:38 INFO Completed upload id=355535worker2 # [ 121.145932] nix-grpc-daemon[1440]: time=2026-09-22T13:47:38.744Z level=INFO msg="Upload complete. (300ms)"5536lb # [ 121.197640] niks3-server[1044]: 2026/09/22 13:47:38 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5537lb # [ 121.207740] niks3-server[1044]: 2026/09/22 13:47:38 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5538worker2 # [ 121.156647] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=0 paths=2 nar_bytes_in=3125539lb # [ 121.222001] niks3-server[1044]: 2026/09/22 13:47:38 INFO Completed upload id=365540worker2 # [ 121.175134] nix-daemon[862]: reaped child process 1438, status = succeeded5541lb # [ 121.236924] niks3-server[1044]: 2026/09/22 13:47:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5542client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.40 seconds)5543lb # [ 121.243440] niks3-server[1044]: 2026/09/22 13:47:38 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5544worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5545worker2 # [ 121.203838] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=scheduler_connected addr=lb:500515546worker2 # [ 121.213271] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5547worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)5548worker1 # [ 121.299136] systemd[1]: Listening on Nix worker-protocol over gRPC.5549worker1 # [ 121.309701] systemd[1]: Starting Nix worker-protocol over gRPC...5550worker2 # [ 121.411267] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:39Z level=info event=scheduler_connected addr=lb:500515551worker2 # [ 121.416844] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5552worker1 # [ 121.459848] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5553worker1 # [ 121.464839] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5554worker1 # [ 121.466796] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15555lb # [ 121.548782] niks3-server[1044]: 2026/09/22 13:47:39 INFO lead: acquired remote=192.168.1.3:426825556worker1 # [ 121.496542] nix-grpc-daemon[1182]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5557lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5558lb: must succeed: curl -sf localhost:9901/clusters5559worker1 # [ 121.498041] nix-grpc-daemon[1182]: E0000 00:00:1790084859.083647 1182 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5560worker1 # [ 121.519847] nix-grpc-daemon[1182]: E0000 00:00:1790084859.085958 1182 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5561worker1 # [ 121.522840] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5562worker1 # [ 121.524608] nix-grpc-daemon[1182]: E0000 00:00:1790084859.094136 1224 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5563worker1 # [ 121.526910] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=scheduler_take_over5564worker1 # [ 121.528097] systemd[1]: Started Nix worker-protocol over gRPC.5565worker1 # [ 121.528961] nix-grpc-daemon[1182]: E0000 00:00:1790084859.115434 1224 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5566worker1 # [ 121.534607] nix-grpc-daemon[1182]: E0000 00:00:1790084859.115617 1224 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5567worker1 # [ 121.539229] nix-grpc-daemon[1182]: E0000 00:00:1790084859.131244 1224 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5568worker1 # [ 121.542166] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=scheduler_connected addr=lb:500515569worker1 # [ 121.543469] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5570lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5571worker2 # [ 121.617985] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:39Z level=info event=scheduler_connected addr=lb:500515572worker2 # [ 121.623439] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5573worker1 # [ 121.743187] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=scheduler_connected addr=lb:500515574worker1 # [ 121.751813] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:344965575worker1 # [ 121.757532] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05576worker2 # [ 121.825869] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:39Z level=info event=scheduler_connected addr=lb:500515577worker1 # [ 121.823690] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=oidc_jwks provider=mock keys=15578worker1 # [ 121.828293] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:344965579worker1 # [ 121.833637] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05580lb: must succeed: curl -sf localhost:9901/clusters5581lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5582lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5583(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.07 seconds)5584subtest: a niks3 restart does not cost the leader its role5585lb: must succeed: curl -sf localhost:9901/clusters5586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5587lb: must succeed: curl -sf localhost:9901/clusters5588lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5589worker2 # [ 122.807919] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:40Z level=info event=scheduler_lock_lost status=2005590worker1 # [ 122.797451] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:40Z level=info event=scheduler_lock_lost status=2005591lb # [ 122.865042] niks3-server[1044]: 2026/09/22 13:47:40 INFO Shutdown signal received, draining in-flight requests timeout=10s5592lb # [ 122.865239] niks3-server[1044]: 2026/09/22 13:47:40 INFO lead: released remote=192.168.1.3:426825593lb # [ 122.884524] systemd[1]: Stopping niks3 server...5594lb # [ 122.892244] systemd[1]: niks3.service: Deactivated successfully.5595lb # [ 122.894791] systemd[1]: Stopped niks3 server.5596lb # [ 122.895555] systemd[1]: niks3.service: Consumed 708ms CPU time over 1min 30.418s wall clock time, 27.8M memory peak, 91.3K incoming IP traffic, 133.5K outgoing IP traffic.5597lb # [ 122.907156] systemd[1]: Starting niks3 server...5598lb # [ 122.976493] niks3-server[1185]: 2026/09/22 13:47:40 goose: no migrations to run. current version: 202609200000005599lb # [ 122.982429] niks3-server[1185]: 2026/09/22 13:47:40 OK 1_commit_pending_closure.sql (7.07ms)5600lb # [ 122.986445] niks3-server[1185]: 2026/09/22 13:47:40 OK 2_object_stats_trigger.sql (3.63ms)5601lb # [ 122.987614] niks3-server[1185]: 2026/09/22 13:47:40 goose: up to current file version: 25602lb # [ 122.989771] niks3-server[1185]: 2026/09/22 13:47:40 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5603lb # [ 123.001023] niks3-server[1185]: 2026/09/22 13:47:40 INFO Read proxy enabled — serving cache objects from S35604lb # [ 123.002403] niks3-server[1185]: 2026/09/22 13:47:40 INFO Using socket-activated listener address=0.0.0.0:57515605lb # [ 123.007621] niks3-server[1185]: 2026/09/22 13:47:40 INFO systemd watchdog enabled interval=15s5606lb # [ 123.009067] niks3-server[1185]: 2026/09/22 13:47:40 INFO Starting HTTP server address=0.0.0.0:57515607lb # [ 123.012678] systemd[1]: Started niks3 server.5608lb: waiting for unit niks3.service5609lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5610lb: must succeed: curl -sf localhost:9901/clusters5611lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5612worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5613worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5614lb: must succeed: curl -sf localhost:9901/clusters5615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5616lb: must succeed: curl -sf localhost:9901/clusters5617lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5618worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5619worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5620client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25621worker1 # [ 123.440910] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:345105622worker1 # [ 123.453158] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5623client # don't know how to build these paths:5624client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5625worker1 # [ 123.480537] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345105626worker1 # [ 123.496724] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)5627worker1 # [ 123.536468] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 2/5)5628worker1 # [ 123.709747] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 128 ms (attempt 3/5)5629lb # [ 123.869130] niks3-server[1185]: 2026/09/22 13:47:41 INFO lead: acquired remote=192.168.1.3:426905630worker1 # [ 123.840770] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 499 ms (attempt 4/5)5631worker1 # [ 124.343305] nix-grpc-daemon[1182]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5632worker1 # [ 124.349409] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5633worker1 # [ 124.404312] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:42Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05634worker1 # [ 124.411028] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:42Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=15635worker1 # [ 124.451424] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345105636worker1 # [ 124.469031] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345105637client # copying 1 paths to 'lb:50051'...5638client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5639worker1 # [ 124.485293] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345105640worker1 # [ 124.500673] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5641worker1 # [ 124.529001] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:42Z level=info event=niks3_push_started pid=12545642worker1 # [ 124.673200] nix-grpc-daemon[1254]: time=2026-09-22T13:47:42.285Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5643worker1 # [ 124.785329] nix-daemon[833]: accepted connection from pid 1267, user nix-grpc-daemon (trusted)5644worker1 # [ 124.809170] nix-daemon[833]: reaped child process 1270, status = succeeded5645worker1 # [ 124.907078] nix-daemon[833]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5646worker1 # [ 124.923256] nix-daemon[833]: reaped child process 1275, status = succeeded5647lb # [ 124.998741] niks3-server[1185]: 2026/09/22 13:47:42 INFO Received uploads request method=POST path=/api/pending_closures5648worker1 # [ 124.952071] nix-grpc-daemon[1254]: time=2026-09-22T13:47:42.563Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5649worker1 # [ 124.953611] nix-grpc-daemon[1254]: time=2026-09-22T13:47:42.564Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5650lb # [ 125.060666] niks3-server[1185]: 2026/09/22 13:47:42 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5651lb # [ 125.074396] niks3-server[1185]: 2026/09/22 13:47:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5652worker1 # [ 125.013070] nix-grpc-daemon[1254]: time=2026-09-22T13:47:42.624Z level=INFO msg="Uploading 1 narinfos"5653lb # [ 125.082013] niks3-server[1185]: 2026/09/22 13:47:42 INFO Signed narinfos id=37 count=15654lb # [ 125.090491] niks3-server[1185]: 2026/09/22 13:47:42 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5655lb # [ 125.100671] niks3-server[1185]: 2026/09/22 13:47:42 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5656lb # [ 125.108805] niks3-server[1185]: 2026/09/22 13:47:42 INFO Completed upload id=375657worker1 # [ 125.042631] nix-grpc-daemon[1254]: time=2026-09-22T13:47:42.654Z level=INFO msg="Upload complete. (364ms)"5658worker1 # [ 125.057796] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34510 duration_s=0 paths=1 nar_bytes_in=5445659worker1 # [ 125.075162] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345105660worker1 # [ 125.079494] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5661worker1 # [ 125.085677] nix-daemon[833]: reaped child process 1252, status = succeeded5662client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5663worker1 # [ 125.097804] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5664client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5665client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5666client # farm-a> LOG-farm-a-after-niks3-restart5667worker1 # [ 126.328083] nix-daemon[833]: reaped child process 1279, status = succeeded5668worker1 # [ 126.424525] nix-daemon[833]: accepted connection from pid 1295, user nix-grpc-daemon (trusted)5669worker1 # [ 126.446515] nix-daemon[833]: reaped child process 1298, status = succeeded5670lb # [ 126.514625] niks3-server[1185]: 2026/09/22 13:47:44 INFO Received uploads request method=POST path=/api/pending_closures5671worker1 # [ 126.464100] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.073Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5672worker1 # [ 126.465599] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.073Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5673lb # [ 126.575684] niks3-server[1185]: 2026/09/22 13:47:44 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5674worker1 # [ 126.518938] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.130Z level=INFO msg="Uploading 1 narinfos"5675lb # [ 126.588540] niks3-server[1185]: 2026/09/22 13:47:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5676lb # [ 126.591513] niks3-server[1185]: 2026/09/22 13:47:44 INFO Signed narinfos id=38 count=15677lb # [ 126.604524] niks3-server[1185]: 2026/09/22 13:47:44 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5678lb # [ 126.607872] niks3-server[1185]: 2026/09/22 13:47:44 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5679worker1 # [ 126.547890] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.160Z level=INFO msg="Upload complete. (240ms)"5680lb # [ 126.617723] niks3-server[1185]: 2026/09/22 13:47:44 INFO Completed upload id=385681lb # [ 126.621201] niks3-server[1185]: 2026/09/22 13:47:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5682lb # [ 126.625018] niks3-server[1185]: 2026/09/22 13:47:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5683worker1 # [ 126.568659] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34510 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15684worker1 # [ 126.581899] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:44Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05685worker1 # [ 126.585310] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:44Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=15686worker1 # [ 126.588921] nix-daemon[833]: reaped child process 1277, status = succeeded5687worker1 # [ 126.609277] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345105688worker1 # [ 126.624978] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345105689worker1 # [ 126.640059] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.252Z level=INFO msg="All 1 paths already cached"5690client # copying 1 paths to 'lb:50051'...5691client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5692worker1 # [ 126.650961] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345105693worker1 # [ 126.665298] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5694worker1 # [ 126.817105] nix-daemon[833]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5695worker1 # [ 126.833594] nix-daemon[833]: reaped child process 1307, status = succeeded5696worker1 # [ 126.938087] nix-daemon[833]: accepted connection from pid 1309, user nix-grpc-daemon (trusted)5697worker1 # [ 126.957289] nix-daemon[833]: reaped child process 1312, status = succeeded5698lb # [ 127.022014] niks3-server[1185]: 2026/09/22 13:47:44 INFO Received uploads request method=POST path=/api/pending_closures5699worker1 # [ 126.966118] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.578Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5700worker1 # [ 126.967589] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.578Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5701lb # [ 127.053686] niks3-server[1185]: 2026/09/22 13:47:44 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5702lb # [ 127.072462] niks3-server[1185]: 2026/09/22 13:47:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5703worker1 # [ 127.013723] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.625Z level=INFO msg="Uploading 1 narinfos"5704lb # [ 127.086510] niks3-server[1185]: 2026/09/22 13:47:44 INFO Signed narinfos id=39 count=15705lb # [ 127.094605] niks3-server[1185]: 2026/09/22 13:47:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5706lb # [ 127.108318] niks3-server[1185]: 2026/09/22 13:47:44 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5707lb # [ 127.115719] niks3-server[1185]: 2026/09/22 13:47:44 INFO Completed upload id=395708worker1 # [ 127.049451] nix-grpc-daemon[1254]: time=2026-09-22T13:47:44.661Z level=INFO msg="Upload complete. (368ms)"5709lb # [ 127.119456] niks3-server[1185]: 2026/09/22 13:47:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5710worker1 # [ 127.060740] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34510 duration_s=0 paths=1 nar_bytes_in=6565711worker1 # [ 127.075722] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345105712worker1 # [ 127.082381] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5713worker1 # [ 127.087414] nix-daemon[833]: reaped child process 1302, status = succeeded5714client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5715worker1 # [ 127.098679] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5716client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5717client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5718client # farm-b> LOG-farm-b-after-niks3-restart5719worker1 # [ 128.376634] nix-daemon[833]: reaped child process 1316, status = succeeded5720worker1 # [ 128.481866] nix-daemon[833]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5721worker1 # [ 128.502721] nix-daemon[833]: reaped child process 1335, status = succeeded5722lb # [ 128.575425] niks3-server[1185]: 2026/09/22 13:47:46 INFO Received uploads request method=POST path=/api/pending_closures5723worker1 # [ 128.520454] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.131Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5724worker1 # [ 128.522448] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.131Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5725lb # [ 128.615235] niks3-server[1185]: 2026/09/22 13:47:46 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5726worker1 # [ 128.564356] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.176Z level=INFO msg="Uploading 1 narinfos"5727lb # [ 128.631086] niks3-server[1185]: 2026/09/22 13:47:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5728lb # [ 128.639261] niks3-server[1185]: 2026/09/22 13:47:46 INFO Signed narinfos id=40 count=15729lb # [ 128.646417] niks3-server[1185]: 2026/09/22 13:47:46 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5730lb # [ 128.652817] niks3-server[1185]: 2026/09/22 13:47:46 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5731lb # [ 128.661480] niks3-server[1185]: 2026/09/22 13:47:46 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5732lb # [ 128.669174] niks3-server[1185]: 2026/09/22 13:47:46 INFO Completed upload id=405733worker1 # [ 128.594574] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.206Z level=INFO msg="Upload complete. (283ms)"5734lb # [ 128.672435] niks3-server[1185]: 2026/09/22 13:47:46 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5735worker1 # [ 128.614143] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34510 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15736worker1 # [ 128.631523] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:46Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05737worker1 # [ 128.635795] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:46Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=15738worker1 # [ 128.640597] nix-daemon[833]: reaped child process 1314, status = succeeded5739worker1 # [ 128.658316] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345105740worker1 # [ 128.673522] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345105741worker1 # [ 128.689265] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.301Z level=INFO msg="All 2 paths already cached"5742client # copying 1 paths to 'lb:50051'...5743client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5744worker1 # [ 128.701678] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345105745worker1 # [ 128.714776] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5746worker1 # [ 128.865297] nix-daemon[833]: accepted connection from pid 1341, user nix-grpc-daemon (trusted)5747worker1 # [ 128.885665] nix-daemon[833]: reaped child process 1344, status = succeeded5748worker1 # [ 128.990721] nix-daemon[833]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5749worker1 # [ 129.008480] nix-daemon[833]: reaped child process 1349, status = succeeded5750lb # [ 129.080064] niks3-server[1185]: 2026/09/22 13:47:46 INFO Received uploads request method=POST path=/api/pending_closures5751worker1 # [ 129.022890] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.634Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5752worker1 # [ 129.024608] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.635Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5753lb # [ 129.110930] niks3-server[1185]: 2026/09/22 13:47:46 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5754worker1 # [ 129.059437] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.671Z level=INFO msg="Uploading 1 narinfos"5755lb # [ 129.127024] niks3-server[1185]: 2026/09/22 13:47:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5756lb # [ 129.137912] niks3-server[1185]: 2026/09/22 13:47:46 INFO Signed narinfos id=41 count=15757lb # [ 129.143532] niks3-server[1185]: 2026/09/22 13:47:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5758lb # [ 129.154805] niks3-server[1185]: 2026/09/22 13:47:46 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5759worker1 # [ 129.094415] nix-grpc-daemon[1254]: time=2026-09-22T13:47:46.706Z level=INFO msg="Upload complete. (363ms)"5760lb # [ 129.162361] niks3-server[1185]: 2026/09/22 13:47:46 INFO Completed upload id=415761lb # [ 129.166713] niks3-server[1185]: 2026/09/22 13:47:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5762worker1 # [ 129.104356] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34510 duration_s=0 paths=1 nar_bytes_in=7845763worker1 # [ 129.118829] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345105764worker1 # [ 129.125454] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5765worker1 # [ 129.130793] nix-daemon[833]: reaped child process 1339, status = succeeded5766client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5767worker1 # [ 129.142719] nix-daemon[833]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5768client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5769lb # [ 130.094855] envoy[913]: conn peer=[2001:db8:1::1]:40046 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1009 tx=282 ms=11645770client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5771client # farm-top> LOG-farm-top-after-niks3-restart5772worker1 # [ 130.420447] nix-daemon[833]: reaped child process 1353, status = succeeded5773worker1 # [ 130.530798] nix-daemon[833]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5774worker1 # [ 130.553830] nix-daemon[833]: reaped child process 1372, status = succeeded5775lb # [ 130.621296] niks3-server[1185]: 2026/09/22 13:47:48 INFO Received uploads request method=POST path=/api/pending_closures5776worker1 # [ 130.564659] nix-grpc-daemon[1254]: time=2026-09-22T13:47:48.176Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5777worker1 # [ 130.568383] nix-grpc-daemon[1254]: time=2026-09-22T13:47:48.176Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5778lb # [ 130.661172] niks3-server[1185]: 2026/09/22 13:47:48 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5779lb # [ 130.673507] niks3-server[1185]: 2026/09/22 13:47:48 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5780worker1 # [ 130.619026] nix-grpc-daemon[1254]: time=2026-09-22T13:47:48.230Z level=INFO msg="Uploading 1 narinfos"5781lb # [ 130.684890] niks3-server[1185]: 2026/09/22 13:47:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5782lb # [ 130.697029] niks3-server[1185]: 2026/09/22 13:47:48 INFO Signed narinfos id=42 count=15783lb # [ 130.703729] niks3-server[1185]: 2026/09/22 13:47:48 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5784lb # [ 130.714497] niks3-server[1185]: 2026/09/22 13:47:48 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5785lb # [ 130.721755] niks3-server[1185]: 2026/09/22 13:47:48 INFO Completed upload id=425786worker1 # [ 130.654746] nix-grpc-daemon[1254]: time=2026-09-22T13:47:48.266Z level=INFO msg="Upload complete. (299ms)"5787lb # [ 130.724431] niks3-server[1185]: 2026/09/22 13:47:48 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5788worker1 # [ 130.669153] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34510 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15789worker1 # [ 130.687307] nix-daemon[833]: reaped child process 1351, status = succeeded5790client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 7.49 seconds)5791client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5792client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5793(finished: subtest: a niks3 restart does not cost the leader its role, in 8.23 seconds)5794subtest: scheduler down: the other node takes the lock and keeps it5795lb: must succeed: curl -sf localhost:9901/clusters5796lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5797lb: must succeed: curl -sf localhost:9901/clusters5798lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5799lb: must succeed: curl -sf localhost:9901/clusters5800lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5801lb: must succeed: curl -sf localhost:9901/clusters5802lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5803worker1 # [ 131.145290] systemd[1]: Stopping Nix worker-protocol over gRPC...5804worker1 # [ 131.478776] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:49Z level=info event=signal_exit5805lb # [ 131.877369] niks3-server[1185]: 2026/09/22 13:47:49 INFO lead: released remote=192.168.1.3:426905806worker2 # [ 132.822339] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:50Z level=info event=scheduler_restarting addr=lb:500515807worker1 # [ 132.817261] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:50Z level=info event=scheduler_restarting5808worker1 # [ 132.821028] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:50Z level=info event=scheduler_restarting addr=lb:500515809worker1 # [ 132.825444] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:344965810worker1 # [ 132.830504] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:50Z level=info event=worker_gone addr=192.168.1.4:500515811worker1 # [ 132.834992] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:344965812worker1 # [ 132.840607] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:50Z level=info event=worker_gone addr=192.168.1.3:500515813worker1 # [ 132.861593] nix-daemon[833]: reaped child process 1242, status = succeeded5814worker2 # [ 133.022245] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:50Z level=info event=scheduler_connected addr=lb:500515815worker2 # [ 133.027593] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5816worker1 # [ 133.030253] nix-grpc-daemon[1182]: ts=2026-09-22T13:47:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5817worker1 # [ 133.104398] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5818worker1 # [ 133.111603] systemd[1]: Stopped Nix worker-protocol over gRPC.5819worker1 # [ 133.117100] systemd[1]: nix-grpc-daemon.service: Consumed 1.486s CPU time over 11.799s wall clock time, 38.4M memory peak, 35.4K incoming IP traffic, 56.9K outgoing IP traffic.5820worker1 # [ 133.135330] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5821worker1 # [ 133.140626] systemd[1]: Closed Nix worker-protocol over gRPC.5822worker1 # [ 133.142743] systemd[1]: nix-grpc-daemon.socket: Consumed 990us CPU time over 11.842s wall clock time, 256K memory peak, 32.9K incoming IP traffic, 26.1K outgoing IP traffic.5823worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5824worker2 # [ 133.228478] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:50Z level=info event=scheduler_connected addr=lb:500515825worker2 # [ 133.236789] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5826worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5827worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5828worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5829worker2 # [ 133.437894] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=scheduler_connected addr=lb:500515830worker2 # [ 133.443067] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5831worker2 # [ 133.644242] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=scheduler_connected addr=lb:500515832worker2 # [ 133.649553] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5833lb # [ 133.877707] niks3-server[1185]: 2026/09/22 13:47:51 INFO lead: acquired remote=192.168.1.4:557805834worker2 # [ 133.822185] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=scheduler_take_over5835worker2 # [ 133.850688] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=scheduler_connected addr=lb:500515836worker2 # [ 133.855870] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5837worker2 # [ 134.056824] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=scheduler_connected addr=lb:500515838worker2 # [ 134.084134] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=oidc_jwks provider=mock keys=15839worker2 # [ 134.088987] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:477305840worker2 # [ 134.094308] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05841worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5842worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5843client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25844worker2 # [ 134.563770] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:350285845client # don't know how to build these paths:5846client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5847worker2 # [ 134.573815] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:350285848worker2 # [ 134.602035] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05849worker2 # [ 134.608952] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15850worker2 # [ 134.648326] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:350285851worker2 # [ 134.662984] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:350285852client # copying 1 paths to 'lb:50051'...5853client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5854worker2 # [ 134.678726] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:350285855worker2 # [ 134.693536] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5856worker2 # [ 134.857716] nix-daemon[862]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)5857worker2 # [ 134.876497] nix-daemon[862]: reaped child process 1482, status = succeeded5858worker2 # [ 134.978909] nix-daemon[862]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)5859worker2 # [ 134.997846] nix-daemon[862]: reaped child process 1487, status = succeeded5860lb # [ 135.058215] niks3-server[1185]: 2026/09/22 13:47:52 INFO Received uploads request method=POST path=/api/pending_closures5861worker2 # [ 135.013812] nix-grpc-daemon[1440]: time=2026-09-22T13:47:52.611Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5862worker2 # [ 135.015339] nix-grpc-daemon[1440]: time=2026-09-22T13:47:52.611Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5863lb # [ 135.088634] niks3-server[1185]: 2026/09/22 13:47:52 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5864worker2 # [ 135.047273] nix-grpc-daemon[1440]: time=2026-09-22T13:47:52.645Z level=INFO msg="Uploading 1 narinfos"5865lb # [ 135.102504] niks3-server[1185]: 2026/09/22 13:47:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5866lb # [ 135.115877] niks3-server[1185]: 2026/09/22 13:47:52 INFO Signed narinfos id=43 count=15867lb # [ 135.123368] niks3-server[1185]: 2026/09/22 13:47:52 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5868lb # [ 135.138644] niks3-server[1185]: 2026/09/22 13:47:52 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5869lb # [ 135.152563] niks3-server[1185]: 2026/09/22 13:47:52 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5870lb # [ 135.175346] niks3-server[1185]: 2026/09/22 13:47:52 INFO Completed upload id=435871worker2 # [ 135.120008] nix-grpc-daemon[1440]: time=2026-09-22T13:47:52.718Z level=INFO msg="Upload complete. (409ms)"5872worker2 # [ 135.132968] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=0 paths=1 nar_bytes_in=5205873worker2 # [ 135.148797] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:350285874worker2 # [ 135.157992] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5875worker2 # [ 135.165086] nix-daemon[862]: reaped child process 1477, status = succeeded5876client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5877worker2 # [ 135.177370] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5878client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5879client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5880client # farm-a> LOG-farm-a-on-standby5881worker2 # [ 136.644219] nix-daemon[862]: reaped child process 1491, status = succeeded5882worker2 # [ 136.775078] nix-daemon[862]: accepted connection from pid 1507, user nix-grpc-daemon (trusted)5883worker2 # [ 136.796166] nix-daemon[862]: reaped child process 1510, status = succeeded5884lb # [ 136.851806] niks3-server[1185]: 2026/09/22 13:47:54 INFO Received uploads request method=POST path=/api/pending_closures5885worker2 # [ 136.816085] nix-grpc-daemon[1440]: time=2026-09-22T13:47:54.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5886worker2 # [ 136.817902] nix-grpc-daemon[1440]: time=2026-09-22T13:47:54.412Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5887lb # [ 136.912911] niks3-server[1185]: 2026/09/22 13:47:54 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5888lb # [ 136.922237] niks3-server[1185]: 2026/09/22 13:47:54 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5889worker2 # [ 136.877971] nix-grpc-daemon[1440]: time=2026-09-22T13:47:54.475Z level=INFO msg="Uploading 1 narinfos"5890lb # [ 136.934395] niks3-server[1185]: 2026/09/22 13:47:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5891lb # [ 136.941155] niks3-server[1185]: 2026/09/22 13:47:54 INFO Signed narinfos id=44 count=15892lb # [ 136.942970] niks3-server[1185]: 2026/09/22 13:47:54 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5893lb # [ 136.953182] niks3-server[1185]: 2026/09/22 13:47:54 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5894lb # [ 136.958736] niks3-server[1185]: 2026/09/22 13:47:54 INFO Completed upload id=445895worker2 # [ 136.902894] nix-grpc-daemon[1440]: time=2026-09-22T13:47:54.501Z level=INFO msg="Upload complete. (309ms)"5896lb # [ 136.961562] niks3-server[1185]: 2026/09/22 13:47:54 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5897worker2 # [ 136.923536] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15898worker2 # [ 136.942949] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:54Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05899worker2 # [ 136.947223] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:54Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15900worker2 # [ 136.951629] nix-daemon[862]: reaped child process 1489, status = succeeded5901worker2 # [ 136.969404] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:350285902worker2 # [ 136.982669] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:350285903worker2 # [ 136.997424] nix-grpc-daemon[1440]: time=2026-09-22T13:47:54.595Z level=INFO msg="All 1 paths already cached"5904client # copying 1 paths to 'lb:50051'...5905client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5906worker2 # [ 137.008070] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:350285907worker2 # [ 137.021589] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5908worker2 # [ 137.172468] nix-daemon[862]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)5909worker2 # [ 137.189246] nix-daemon[862]: reaped child process 1519, status = succeeded5910worker2 # [ 137.294810] nix-daemon[862]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)5911worker2 # [ 137.314320] nix-daemon[862]: reaped child process 1524, status = succeeded5912lb # [ 137.373960] niks3-server[1185]: 2026/09/22 13:47:54 INFO Received uploads request method=POST path=/api/pending_closures5913worker2 # [ 137.328247] nix-grpc-daemon[1440]: time=2026-09-22T13:47:54.926Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5914worker2 # [ 137.330402] nix-grpc-daemon[1440]: time=2026-09-22T13:47:54.926Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5915lb # [ 137.405489] niks3-server[1185]: 2026/09/22 13:47:55 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5916lb # [ 137.421003] niks3-server[1185]: 2026/09/22 13:47:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5917worker2 # [ 137.371066] nix-grpc-daemon[1440]: time=2026-09-22T13:47:54.969Z level=INFO msg="Uploading 1 narinfos"5918lb # [ 137.431806] niks3-server[1185]: 2026/09/22 13:47:55 INFO Signed narinfos id=45 count=15919lb # [ 137.438806] niks3-server[1185]: 2026/09/22 13:47:55 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5920lb # [ 137.450449] niks3-server[1185]: 2026/09/22 13:47:55 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5921worker2 # [ 137.401618] nix-grpc-daemon[1440]: time=2026-09-22T13:47:55.000Z level=INFO msg="Upload complete. (364ms)"5922lb # [ 137.459219] niks3-server[1185]: 2026/09/22 13:47:55 INFO Completed upload id=455923lb # [ 137.463201] niks3-server[1185]: 2026/09/22 13:47:55 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5924worker2 # [ 137.412364] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=0 paths=1 nar_bytes_in=6405925worker2 # [ 137.427169] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:350285926worker2 # [ 137.433892] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5927worker2 # [ 137.438792] nix-daemon[862]: reaped child process 1514, status = succeeded5928worker2 # [ 137.450022] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5929client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5930client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5931client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5932client # farm-b> LOG-farm-b-on-standby5933worker2 # [ 138.508354] nix-daemon[862]: reaped child process 1528, status = succeeded5934worker2 # [ 138.614645] nix-daemon[862]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)5935worker2 # [ 138.634587] nix-daemon[862]: reaped child process 1547, status = succeeded5936lb # [ 138.695669] niks3-server[1185]: 2026/09/22 13:47:56 INFO Received uploads request method=POST path=/api/pending_closures5937worker2 # [ 138.650758] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.248Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5938worker2 # [ 138.652526] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.248Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5939lb # [ 138.735399] niks3-server[1185]: 2026/09/22 13:47:56 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5940lb # [ 138.746715] niks3-server[1185]: 2026/09/22 13:47:56 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5941worker2 # [ 138.703101] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.301Z level=INFO msg="Uploading 1 narinfos"5942lb # [ 138.758226] niks3-server[1185]: 2026/09/22 13:47:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5943lb # [ 138.770194] niks3-server[1185]: 2026/09/22 13:47:56 INFO Signed narinfos id=46 count=15944lb # [ 138.776610] niks3-server[1185]: 2026/09/22 13:47:56 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5945lb # [ 138.790158] niks3-server[1185]: 2026/09/22 13:47:56 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5946worker2 # [ 138.733976] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.332Z level=INFO msg="Upload complete. (272ms)"5947lb # [ 138.795182] niks3-server[1185]: 2026/09/22 13:47:56 INFO Completed upload id=465948lb # [ 138.798806] niks3-server[1185]: 2026/09/22 13:47:56 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5949worker2 # [ 138.751688] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15950worker2 # [ 138.772438] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:56Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05951worker2 # [ 138.778355] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:56Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15952worker2 # [ 138.783833] nix-daemon[862]: reaped child process 1526, status = succeeded5953worker2 # [ 138.812905] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:350285954worker2 # [ 138.828555] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:350285955worker2 # [ 138.844346] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.442Z level=INFO msg="All 2 paths already cached"5956client # copying 1 paths to 'lb:50051'...5957client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5958worker2 # [ 138.855740] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:350285959worker2 # [ 138.869880] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5960worker2 # [ 139.024066] nix-daemon[862]: accepted connection from pid 1553, user nix-grpc-daemon (trusted)5961worker2 # [ 139.048705] nix-daemon[862]: reaped child process 1556, status = succeeded5962worker2 # [ 139.149413] nix-daemon[862]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)5963worker2 # [ 139.171258] nix-daemon[862]: reaped child process 1561, status = succeeded5964lb # [ 139.231336] niks3-server[1185]: 2026/09/22 13:47:56 INFO Received uploads request method=POST path=/api/pending_closures5965worker2 # [ 139.186078] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.784Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5966worker2 # [ 139.187816] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.784Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5967lb # [ 139.265377] niks3-server[1185]: 2026/09/22 13:47:56 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5968lb # [ 139.283367] niks3-server[1185]: 2026/09/22 13:47:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5969worker2 # [ 139.232671] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.830Z level=INFO msg="Uploading 1 narinfos"5970lb # [ 139.294684] niks3-server[1185]: 2026/09/22 13:47:56 INFO Signed narinfos id=47 count=15971lb # [ 139.300804] niks3-server[1185]: 2026/09/22 13:47:56 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5972lb # [ 139.311851] niks3-server[1185]: 2026/09/22 13:47:56 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5973worker2 # [ 139.261702] nix-grpc-daemon[1440]: time=2026-09-22T13:47:56.860Z level=INFO msg="Upload complete. (375ms)"5974lb # [ 139.318820] niks3-server[1185]: 2026/09/22 13:47:56 INFO Completed upload id=475975lb # [ 139.321745] niks3-server[1185]: 2026/09/22 13:47:56 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5976worker2 # [ 139.272338] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=0 paths=1 nar_bytes_in=7685977worker2 # [ 139.288443] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:350285978worker2 # [ 139.294768] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5979worker2 # [ 139.299800] nix-daemon[862]: reaped child process 1551, status = succeeded5980worker2 # [ 139.311241] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5981client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5982client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5983lb # [ 140.096883] envoy[913]: conn peer=[2001:db8:1::1]:33184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9727 tx=4463 ms=72455984lb # [ 140.101953] envoy[913]: conn peer=[2001:db8:1::3]:36086 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1200 tx=513 ms=114915985client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5986client # farm-top> LOG-farm-top-on-standby5987worker2 # [ 140.789000] nix-daemon[862]: reaped child process 1565, status = succeeded5988worker2 # [ 140.894681] nix-daemon[862]: accepted connection from pid 1581, user nix-grpc-daemon (trusted)5989worker2 # [ 140.915386] nix-daemon[862]: reaped child process 1584, status = succeeded5990lb # [ 140.978684] niks3-server[1185]: 2026/09/22 13:47:58 INFO Received uploads request method=POST path=/api/pending_closures5991worker2 # [ 140.932598] nix-grpc-daemon[1440]: time=2026-09-22T13:47:58.530Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5992worker2 # [ 140.937132] nix-grpc-daemon[1440]: time=2026-09-22T13:47:58.530Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5993lb # [ 141.030845] niks3-server[1185]: 2026/09/22 13:47:58 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5994lb # [ 141.041219] niks3-server[1185]: 2026/09/22 13:47:58 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5995lb # [ 141.050377] niks3-server[1185]: 2026/09/22 13:47:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5996worker2 # [ 140.994864] nix-grpc-daemon[1440]: time=2026-09-22T13:47:58.592Z level=INFO msg="Uploading 1 narinfos"5997lb # [ 141.058285] niks3-server[1185]: 2026/09/22 13:47:58 INFO Signed narinfos id=48 count=15998lb # [ 141.063528] niks3-server[1185]: 2026/09/22 13:47:58 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5999lb # [ 141.072387] niks3-server[1185]: 2026/09/22 13:47:58 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6000lb # [ 141.078288] niks3-server[1185]: 2026/09/22 13:47:58 INFO Completed upload id=486001worker2 # [ 141.021457] nix-grpc-daemon[1440]: time=2026-09-22T13:47:58.619Z level=INFO msg="Upload complete. (297ms)"6002lb # [ 141.081182] niks3-server[1185]: 2026/09/22 13:47:58 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo6003worker2 # [ 141.037096] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=16004worker2 # [ 141.052967] nix-daemon[862]: reaped child process 1563, status = succeeded6005client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.73 seconds)6006client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath6007client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)6008worker1 # [ 141.325208] systemd[1]: Listening on Nix worker-protocol over gRPC.6009worker1 # [ 141.334956] systemd[1]: Starting Nix worker-protocol over gRPC...6010worker1 # [ 141.478483] nix-grpc-daemon[1390]: ts=2026-09-22T13:47:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6011worker1 # [ 141.483662] nix-grpc-daemon[1390]: ts=2026-09-22T13:47:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6012worker1 # [ 141.508536] nix-grpc-daemon[1390]: ts=2026-09-22T13:47:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16013worker1 # [ 141.510792] nix-grpc-daemon[1390]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6014lb: must succeed: curl -sf localhost:9901/clusters6015worker1 # [ 141.534895] nix-grpc-daemon[1390]: E0000 00:00:1790084879.099717 1390 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6016worker1 # [ 141.538020] nix-grpc-daemon[1390]: E0000 00:00:1790084879.102024 1390 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6017worker1 # [ 141.540561] nix-grpc-daemon[1390]: ts=2026-09-22T13:47:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6018worker1 # [ 141.542213] nix-grpc-daemon[1390]: E0000 00:00:1790084879.110487 1429 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6019worker1 # [ 141.544650] systemd[1]: Started Nix worker-protocol over gRPC.6020worker1 # [ 141.545481] nix-grpc-daemon[1390]: E0000 00:00:1790084879.130618 1429 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6021worker1 # [ 141.550962] nix-grpc-daemon[1390]: E0000 00:00:1790084879.130775 1429 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6022worker1 # [ 141.555529] nix-grpc-daemon[1390]: E0000 00:00:1790084879.146429 1429 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6023worker1 # [ 141.558115] nix-grpc-daemon[1390]: ts=2026-09-22T13:47:59Z level=info event=scheduler_connected addr=lb:500516024worker2 # [ 141.566846] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:477306025worker2 # [ 141.573567] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06026lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6027worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6028worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6029lb: must succeed: curl -sf localhost:9901/clusters6030lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6031lb: must succeed: curl -sf localhost:9901/clusters6032lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6033worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6034worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6035(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.96 seconds)6036subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6037client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs6038client # [ 141.973464] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.6039client # Running as unit: rs.service; invocation ID: 2a2814fac293445ab67be4967299f3bd6040client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)6041worker1 # [ 142.125213] nix-grpc-daemon[1390]: ts=2026-09-22T13:47:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:344566042worker1 # [ 142.139779] nix-daemon[833]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6043client # [ 142.209500] nix[1344]: don't know how to build these paths:6044client # [ 142.212092] nix[1344]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv6045worker2 # [ 142.182217] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:59Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=06046worker2 # [ 142.182578] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:59Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=16047worker2 # [ 142.225356] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:350286048worker2 # [ 142.241130] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:350286049client # [ 142.296158] nix[1344]: copying 1 paths to 'lb:50051'...6050worker2 # [ 142.255727] nix-grpc-daemon[1380]: ts=2026-09-22T13:47:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:350286051client # [ 142.298469] nix[1344]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...6052worker2 # [ 142.272423] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)6053worker2 # [ 142.438483] nix-daemon[862]: accepted connection from pid 1597, user nix-grpc-daemon (trusted)6054worker2 # [ 142.457383] nix-daemon[862]: reaped child process 1600, status = succeeded6055worker2 # [ 142.558534] nix-daemon[862]: accepted connection from pid 1602, user nix-grpc-daemon (trusted)6056worker2 # [ 142.578981] nix-daemon[862]: reaped child process 1605, status = succeeded6057lb # [ 142.633510] niks3-server[1185]: 2026/09/22 13:48:00 INFO Received uploads request method=POST path=/api/pending_closures6058worker2 # [ 142.596092] nix-grpc-daemon[1440]: time=2026-09-22T13:48:00.193Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6059worker2 # [ 142.597728] nix-grpc-daemon[1440]: time=2026-09-22T13:48:00.193Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"6060lb # [ 142.662879] niks3-server[1185]: 2026/09/22 13:48:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6061lb # [ 142.668908] niks3-server[1185]: 2026/09/22 13:48:00 INFO Signed narinfos id=49 count=16062worker2 # [ 142.613301] nix-grpc-daemon[1440]: time=2026-09-22T13:48:00.211Z level=INFO msg="Uploading 1 narinfos"6063lb # [ 142.678259] niks3-server[1185]: 2026/09/22 13:48:00 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls6064lb # [ 142.688171] niks3-server[1185]: 2026/09/22 13:48:00 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6065worker2 # [ 142.637098] nix-grpc-daemon[1440]: time=2026-09-22T13:48:00.235Z level=INFO msg="Upload complete. (347ms)"6066lb # [ 142.695294] niks3-server[1185]: 2026/09/22 13:48:00 INFO Completed upload id=496067worker2 # [ 142.641516] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=0 paths=1 nar_bytes_in=4166068lb # [ 142.700653] niks3-server[1185]: 2026/09/22 13:48:00 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo6069lb # [ 142.704350] niks3-server[1185]: 2026/09/22 13:48:00 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst6070worker2 # [ 142.649748] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:350286071worker2 # [ 142.655448] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)6072worker2 # [ 142.656805] nix-daemon[862]: reaped child process 1595, status = succeeded6073client # [ 142.704438] nix[1344]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...6074worker2 # [ 142.665979] nix-daemon[862]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)6075lb: must succeed: curl -sf localhost:9901/clusters6076lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6077lb: must succeed: curl -sf localhost:9901/clusters6078lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6079lb: must succeed: curl -sf localhost:9901/clusters6080lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6081lb: must succeed: curl -sf localhost:9901/clusters6082lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6083worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6084worker1 # [ 144.800850] systemd-journald[268]: Received client request to rotate journal, rotating.6085worker1 # [ 144.823629] systemd-journald[268]: Vacuuming done, freed 0B of archived journals from /var/log/journal/2ed7ad5c12694ae7a6410b3841c97a6b.6086worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)6087worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6088worker2 # [ 144.889018] systemd[1]: Stopping Nix worker-protocol over gRPC...6089worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)6090client: must succeed: sleep 5.06091worker2 # [ 145.002078] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:02Z level=info event=draining6092client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6093lb # [ 150.096795] envoy[913]: conn peer=[2001:db8:1::1]:43914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9623 tx=4413 ms=64846094worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6095worker2 # [ 150.310377] nix-daemon[862]: reaped child process 1609, status = succeeded6096worker2 # [ 150.434059] nix-daemon[862]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)6097worker2 # [ 150.456151] nix-daemon[862]: reaped child process 1653, status = succeeded6098lb # [ 150.511260] niks3-server[1185]: 2026/09/22 13:48:08 INFO Received uploads request method=POST path=/api/pending_closures6099worker2 # [ 150.474512] nix-grpc-daemon[1440]: time=2026-09-22T13:48:08.070Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6100worker2 # [ 150.476270] nix-grpc-daemon[1440]: time=2026-09-22T13:48:08.070Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"6101lb # [ 150.570118] niks3-server[1185]: 2026/09/22 13:48:08 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv6102lb # [ 150.580295] niks3-server[1185]: 2026/09/22 13:48:08 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6103worker2 # [ 150.535408] nix-grpc-daemon[1440]: time=2026-09-22T13:48:08.133Z level=INFO msg="Uploading 1 narinfos"6104lb # [ 150.591510] niks3-server[1185]: 2026/09/22 13:48:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6105lb # [ 150.599997] niks3-server[1185]: 2026/09/22 13:48:08 INFO Signed narinfos id=50 count=16106lb # [ 150.602094] niks3-server[1185]: 2026/09/22 13:48:08 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls6107lb # [ 150.614702] niks3-server[1185]: 2026/09/22 13:48:08 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6108lb # [ 150.621259] niks3-server[1185]: 2026/09/22 13:48:08 INFO Completed upload id=506109worker2 # [ 150.565936] nix-grpc-daemon[1440]: time=2026-09-22T13:48:08.164Z level=INFO msg="Upload complete. (286ms)"6110lb # [ 150.623674] niks3-server[1185]: 2026/09/22 13:48:08 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo6111worker2 # [ 150.581354] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35028 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=16112worker2 # [ 150.601218] nix-daemon[862]: reaped child process 1607, status = succeeded6113client # [ 150.652489] systemd[1]: rs.service: Deactivated successfully.6114client # [ 150.657653] systemd[1]: rs.service: Consumed 197ms CPU time over 8.682s wall clock time, 41.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6115worker2 # [ 151.003936] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:08Z level=info event=signal_exit6116lb # [ 151.881885] niks3-server[1185]: 2026/09/22 13:48:09 INFO lead: released remote=192.168.1.4:557806117worker1 # [ 152.817400] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:10Z level=info event=scheduler_restarting addr=lb:500516118worker2 # [ 152.831708] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:10Z level=info event=scheduler_restarting6119worker2 # [ 152.837111] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:10Z level=info event=scheduler_restarting addr=lb:500516120worker2 # [ 152.841582] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477306121worker2 # [ 152.846783] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:10Z level=info event=worker_gone addr=192.168.1.4:500516122worker2 # [ 152.851032] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477306123worker2 # [ 152.855588] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:10Z level=info event=worker_gone addr=192.168.1.3:500516124worker2 # [ 152.882762] nix-daemon[862]: reaped child process 1432, status = succeeded6125worker1 # [ 153.017577] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:10Z level=info event=scheduler_connected addr=lb:500516126worker1 # [ 153.023457] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6127worker2 # [ 153.050304] nix-grpc-daemon[1380]: ts=2026-09-22T13:48:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6128worker2 # [ 153.098873] nix-grpc-daemon[1380]: E0000 00:00:1790084890.696630 1425 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6129worker2 # [ 153.105676] nix-grpc-daemon[1380]: E0000 00:00:1790084890.697078 1425 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6130worker1 # [ 153.224257] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:10Z level=info event=scheduler_connected addr=lb:500516131worker1 # [ 153.229678] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6132worker1 # [ 153.430586] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_connected addr=lb:500516133worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.27 seconds)6134client: waiting for success: ! systemctl is-active rs6135worker1 # [ 153.436610] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6136client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)6137client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6138client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)6139client: must fail: journalctl -u rs | grep -q 'reconnecting'6140client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6141lb: must succeed: curl -sf localhost:9901/clusters6142worker1 # [ 153.637258] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_connected addr=lb:500516143worker1 # [ 153.642801] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6144lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6145worker2 # [ 153.720972] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6146worker2 # [ 153.727557] systemd[1]: Stopped Nix worker-protocol over gRPC.6147worker2 # [ 153.732524] systemd[1]: nix-grpc-daemon.service: Consumed 2.152s CPU time over 36.891s wall clock time, 38.8M memory peak, 59K incoming IP traffic, 131.2K outgoing IP traffic.6148worker2 # [ 153.751048] systemd[1]: Starting scheduler-token.service...6149worker2 # [ 153.830215] systemd[1]: scheduler-token.service: Deactivated successfully.6150worker2 # [ 153.835676] systemd[1]: Finished scheduler-token.service.6151worker2 # [ 153.840507] systemd[1]: scheduler-token.service: Consumed 48ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6152worker2 # [ 153.852199] systemd[1]: Starting Nix worker-protocol over gRPC...6153worker1 # [ 153.845507] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_connected addr=lb:500516154worker1 # [ 153.850407] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6155worker2 # [ 153.981977] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6156worker2 # [ 153.984331] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6157worker2 # [ 153.988622] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16158worker2 # [ 153.996037] nix-grpc-daemon[1663]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6159worker2 # [ 153.997502] nix-grpc-daemon[1663]: E0000 00:00:1790084891.592068 1663 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6160worker2 # [ 153.999782] nix-grpc-daemon[1663]: E0000 00:00:1790084891.594315 1663 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6161lb # [ 154.066640] niks3-server[1185]: 2026/09/22 13:48:11 INFO lead: acquired remote=192.168.1.4:366266162worker2 # [ 154.039858] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6163worker2 # [ 154.042238] nix-grpc-daemon[1663]: E0000 00:00:1790084891.606874 1702 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6164worker2 # [ 154.044827] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=scheduler_take_over6165worker2 # [ 154.045937] nix-grpc-daemon[1663]: E0000 00:00:1790084891.622387 1702 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6166worker2 # [ 154.051390] nix-grpc-daemon[1663]: E0000 00:00:1790084891.622553 1702 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6167worker2 # [ 154.056411] nix-grpc-daemon[1663]: E0000 00:00:1790084891.637366 1702 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6168worker2 # [ 154.058907] systemd[1]: Started Nix worker-protocol over gRPC.6169worker2 # [ 154.059743] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=scheduler_connected addr=lb:500516170worker1 # [ 154.046199] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_connected addr=lb:500516171worker2 # [ 154.061129] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6172worker1 # [ 154.051658] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6173worker2 # [ 154.258156] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=scheduler_connected addr=lb:500516174worker1 # [ 154.252629] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_connected addr=lb:500516175worker2 # [ 154.263339] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6176worker1 # [ 154.258088] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6177worker2 # [ 154.464844] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=scheduler_connected addr=lb:500516178worker1 # [ 154.459498] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:12Z level=info event=scheduler_connected addr=lb:500516179worker2 # [ 154.470358] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6180worker1 # [ 154.464767] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6181lb: must succeed: curl -sf localhost:9901/clusters6182worker2 # [ 154.671636] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=scheduler_connected addr=lb:500516183worker1 # [ 154.661618] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:12Z level=info event=scheduler_connected addr=lb:500516184worker2 # [ 154.677462] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6185worker1 # [ 154.667308] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6186lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6187worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6188worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6189worker2 # [ 154.878669] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=scheduler_connected addr=lb:500516190worker1 # [ 154.869254] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:12Z level=info event=scheduler_connected addr=lb:500516191worker2 # [ 154.888162] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:400346192worker2 # [ 154.895237] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06193worker2 # [ 154.907692] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=oidc_jwks provider=mock keys=16194worker2 # [ 154.912337] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:400346195worker2 # [ 154.917618] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06196lb: must succeed: curl -sf localhost:9901/clusters6197lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6198worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6199worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6200(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 14.10 seconds)6201subtest: scheduler restart mid-build: build finishes, no double build6202client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid6203client # [ 156.068982] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.6204client # Running as unit: mid.service; invocation ID: 00fcacfbd465403b9fea32ffb759438c6205client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)6206worker2 # [ 156.231664] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400386207worker2 # [ 156.244591] nix-daemon[862]: accepted connection from pid 1663, user nix-grpc-daemon (trusted)6208client # [ 156.301505] nix[1380]: don't know how to build these paths:6209client # [ 156.302865] nix[1380]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6210worker2 # [ 156.274780] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:13Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06211worker2 # [ 156.281767] nix-grpc-daemon[1663]: ts=2026-09-22T13:48:13Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16212worker1 # [ 156.319775] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344566213worker1 # [ 156.336568] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)6214worker1 # [ 156.360138] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 2/5)6215worker1 # [ 156.528294] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 3/5)6216worker1 # [ 156.580477] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 463 ms (attempt 4/5)6217worker1 # [ 157.046296] nix-grpc-daemon[1390]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6218worker1 # [ 157.053220] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6219worker1 # [ 157.073865] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344566220client # [ 157.140464] nix[1380]: copying 1 paths to 'lb:50051'...6221worker1 # [ 157.090756] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344566222client # [ 157.144155] nix[1380]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...6223worker1 # [ 157.108999] nix-daemon[833]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6224worker1 # [ 157.139057] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:14Z level=info event=niks3_push_started pid=15106225worker1 # [ 157.312705] nix-grpc-daemon[1510]: time=2026-09-22T13:48:14.921Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6226worker1 # [ 157.429838] nix-daemon[833]: accepted connection from pid 1527, user nix-grpc-daemon (trusted)6227worker1 # [ 157.451043] nix-daemon[833]: reaped child process 1530, status = succeeded6228worker1 # [ 157.555580] nix-daemon[833]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)6229worker1 # [ 157.575136] nix-daemon[833]: reaped child process 1535, status = succeeded6230lb # [ 157.647410] niks3-server[1185]: 2026/09/22 13:48:15 INFO Received uploads request method=POST path=/api/pending_closures6231worker1 # [ 157.595028] nix-grpc-daemon[1510]: time=2026-09-22T13:48:15.206Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6232worker1 # [ 157.597139] nix-grpc-daemon[1510]: time=2026-09-22T13:48:15.206Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"6233lb # [ 157.693326] niks3-server[1185]: 2026/09/22 13:48:15 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst6234worker1 # [ 157.638726] nix-grpc-daemon[1510]: time=2026-09-22T13:48:15.250Z level=INFO msg="Uploading 1 narinfos"6235lb # [ 157.709420] niks3-server[1185]: 2026/09/22 13:48:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6236lb # [ 157.718461] niks3-server[1185]: 2026/09/22 13:48:15 INFO Signed narinfos id=51 count=16237lb # [ 157.727252] niks3-server[1185]: 2026/09/22 13:48:15 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6238lb # [ 157.740234] niks3-server[1185]: 2026/09/22 13:48:15 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls6239worker1 # [ 157.679204] nix-grpc-daemon[1510]: time=2026-09-22T13:48:15.291Z level=INFO msg="Upload complete. (369ms)"6240lb # [ 157.748240] niks3-server[1185]: 2026/09/22 13:48:15 INFO Completed upload id=516241worker1 # [ 157.689487] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34456 duration_s=0 paths=1 nar_bytes_in=4246242worker1 # [ 157.704473] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344566243worker1 # [ 157.713125] nix-daemon[833]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6244worker1 # [ 157.717768] nix-daemon[833]: reaped child process 1508, status = succeeded6245worker1 # [ 157.728626] nix-daemon[833]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6246client # [ 157.778103] nix[1380]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6247lb: must succeed: curl -sf localhost:9901/clusters6248lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6249lb: must succeed: curl -sf localhost:9901/clusters6250lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6251worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6252worker2 # [ 159.997599] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1663 (nix-grpc-daemon) on client request.6253worker2 # [ 160.016751] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6254worker1 # [ 160.025541] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:17Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6255lb # [ 160.089187] niks3-server[1185]: 2026/09/22 13:48:17 INFO lead: released remote=192.168.1.4:366266256client # [ 160.078058] nix[1380]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6257lb # [ 160.097316] envoy[913]: conn peer=[2001:db8:1::1]:33460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1543 ms=84596258worker2 # [ 160.049069] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6259lb # [ 160.102312] envoy[913]: conn peer=[2001:db8:1::4]:47754 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=17996 tx=1290 ms=359886260worker2 # [ 160.051546] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6261lb # [ 160.107645] envoy[913]: conn peer=[2001:db8:1::4]:57086 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4102 tx=385 ms=59926262worker2 # [ 160.071306] systemd[1]: nix-grpc-daemon.service: Consumed 235ms CPU time over 6.189s wall clock time, 14.4M memory peak, 9.8K incoming IP traffic, 16.6K outgoing IP traffic.6263worker2 # [ 160.080840] systemd[1]: Starting scheduler-token.service...6264worker2 # [ 160.083525] nix-daemon[862]: reaped child process 1725, status = succeeded6265worker2 # [ 160.144366] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6266worker2 # [ 160.150632] systemd[1]: scheduler-token.service: Deactivated successfully.6267worker2 # [ 160.155627] systemd[1]: Finished scheduler-token.service.6268worker2 # [ 160.158987] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 73ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6269worker2 # [ 160.172118] systemd[1]: Starting Nix worker-protocol over gRPC...6270worker1 # [ 160.225644] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:17Z level=info event=scheduler_connected addr=lb:500516271worker2 # [ 160.275383] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6272worker2 # [ 160.277764] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6273worker2 # [ 160.281533] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16274worker2 # [ 160.288066] nix-grpc-daemon[1756]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6275worker2 # [ 160.289537] nix-grpc-daemon[1756]: E0000 00:00:1790084897.884438 1756 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6276worker2 # [ 160.291856] nix-grpc-daemon[1756]: E0000 00:00:1790084897.886484 1756 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6277worker2 # [ 160.304344] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6278lb # [ 160.360720] niks3-server[1185]: 2026/09/22 13:48:17 INFO lead: acquired remote=192.168.1.4:366326279worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.38 seconds)6280client: must succeed: sleep 5.06281worker2 # [ 160.342590] systemd[1]: Started Nix worker-protocol over gRPC.6282worker2 # [ 160.344152] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=scheduler_take_over6283worker2 # [ 160.345452] nix-grpc-daemon[1756]: E0000 00:00:1790084897.906809 1795 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6284worker2 # [ 160.347750] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:400426285worker2 # [ 160.352167] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=16286worker2 # [ 160.356569] nix-grpc-daemon[1756]: E0000 00:00:1790084897.925369 1795 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6287worker2 # [ 160.359331] nix-grpc-daemon[1756]: E0000 00:00:1790084897.925521 1795 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6288worker2 # [ 160.362396] nix-grpc-daemon[1756]: E0000 00:00:1790084897.940254 1795 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6289worker2 # [ 160.365247] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=scheduler_connected addr=lb:500516290worker2 # [ 160.368550] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=oidc_jwks provider=mock keys=16291worker2 # [ 160.369816] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:400426292worker2 # [ 160.371497] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06293client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6294client: waiting for success: ! systemctl is-active mid6295worker1 # [ 165.829375] nix-daemon[833]: reaped child process 1539, status = succeeded6296worker1 # [ 165.919194] nix-daemon[833]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6297worker1 # [ 165.938838] nix-daemon[833]: reaped child process 1579, status = succeeded6298lb # [ 166.015016] niks3-server[1185]: 2026/09/22 13:48:23 INFO Received uploads request method=POST path=/api/pending_closures6299worker1 # [ 165.960499] nix-grpc-daemon[1510]: time=2026-09-22T13:48:23.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6300worker1 # [ 165.962391] nix-grpc-daemon[1510]: time=2026-09-22T13:48:23.571Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6301lb # [ 166.055723] niks3-server[1185]: 2026/09/22 13:48:23 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6302lb # [ 166.074109] niks3-server[1185]: 2026/09/22 13:48:23 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6303lb # [ 166.087190] niks3-server[1185]: 2026/09/22 13:48:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6304lb # [ 166.092666] niks3-server[1185]: 2026/09/22 13:48:23 INFO Signed narinfos id=52 count=16305worker1 # [ 166.026972] nix-grpc-daemon[1510]: time=2026-09-22T13:48:23.637Z level=INFO msg="Uploading 1 narinfos"6306lb # [ 166.098664] niks3-server[1185]: 2026/09/22 13:48:23 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6307lb # [ 166.111806] niks3-server[1185]: 2026/09/22 13:48:23 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6308lb # [ 166.118951] niks3-server[1185]: 2026/09/22 13:48:23 INFO Completed upload id=526309lb # [ 166.121420] niks3-server[1185]: 2026/09/22 13:48:23 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6310worker1 # [ 166.050544] nix-grpc-daemon[1510]: time=2026-09-22T13:48:23.662Z level=INFO msg="Upload complete. (306ms)"6311worker1 # [ 166.064440] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34456 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16312worker1 # [ 166.082590] nix-daemon[833]: reaped child process 1537, status = succeeded6313client # [ 166.148822] systemd[1]: mid.service: Deactivated successfully.6314client # [ 166.153577] systemd[1]: mid.service: Consumed 196ms CPU time over 10.081s wall clock time, 41.4M memory peak, 7.8K incoming IP traffic, 9K outgoing IP traffic.6315client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6316client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6317client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)6318(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.85 seconds)6319subtest: interrupting the client stops the build on the worker6320client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6321client # [ 166.892629] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6322client # Running as unit: intr.service; invocation ID: 60c250a645da44b9847c678a1b5209846323client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6324worker1 # [ 167.018168] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:344566325client # [ 167.078824] nix[1416]: don't know how to build these paths:6326client # [ 167.079776] nix[1416]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6327worker2 # [ 167.051180] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:24Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06328worker2 # [ 167.056868] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:24Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16329worker2 # [ 167.097946] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537186330worker2 # [ 167.112629] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6331worker2 # [ 167.134913] nix-grpc-daemon[1756]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 1/5)6332worker2 # [ 167.213918] nix-grpc-daemon[1756]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 2/5)6333worker2 # [ 167.316226] nix-grpc-daemon[1756]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 3/5)6334worker2 # [ 167.347692] nix-grpc-daemon[1756]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 261 ms (attempt 4/5)6335worker2 # [ 167.611768] nix-grpc-daemon[1756]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6336worker2 # [ 167.617813] nix-grpc-daemon[1756]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6337worker2 # [ 167.636979] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537186338client # [ 167.692331] nix[1416]: copying 1 paths to 'lb:50051'...6339worker2 # [ 167.652929] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537186340client # [ 167.696208] nix[1416]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6341worker2 # [ 167.670222] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6342worker2 # [ 167.696644] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:25Z level=info event=niks3_push_started pid=18306343worker2 # [ 167.845091] nix-grpc-daemon[1830]: time=2026-09-22T13:48:25.443Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6344worker2 # [ 167.957480] nix-daemon[862]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6345worker2 # [ 167.980347] nix-daemon[862]: reaped child process 1845, status = succeeded6346worker2 # [ 168.080731] nix-daemon[862]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6347worker2 # [ 168.105796] nix-daemon[862]: reaped child process 1850, status = succeeded6348lb # [ 168.157497] niks3-server[1185]: 2026/09/22 13:48:25 INFO Received uploads request method=POST path=/api/pending_closures6349worker2 # [ 168.117522] nix-grpc-daemon[1830]: time=2026-09-22T13:48:25.714Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6350worker2 # [ 168.119095] nix-grpc-daemon[1830]: time=2026-09-22T13:48:25.714Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6351lb # [ 168.200633] niks3-server[1185]: 2026/09/22 13:48:25 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6352worker2 # [ 168.159893] nix-grpc-daemon[1830]: time=2026-09-22T13:48:25.758Z level=INFO msg="Uploading 1 narinfos"6353lb # [ 168.216182] niks3-server[1185]: 2026/09/22 13:48:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6354lb # [ 168.225585] niks3-server[1185]: 2026/09/22 13:48:25 INFO Signed narinfos id=53 count=16355lb # [ 168.232386] niks3-server[1185]: 2026/09/22 13:48:25 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6356worker2 # [ 168.186548] nix-grpc-daemon[1830]: time=2026-09-22T13:48:25.785Z level=INFO msg="Upload complete. (337ms)"6357lb # [ 168.242444] niks3-server[1185]: 2026/09/22 13:48:25 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6358worker2 # [ 168.192630] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=0 paths=1 nar_bytes_in=4246359lb # [ 168.252519] niks3-server[1185]: 2026/09/22 13:48:25 INFO Completed upload id=536360lb # [ 168.255930] niks3-server[1185]: 2026/09/22 13:48:25 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6361worker2 # [ 168.203066] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537186362worker2 # [ 168.208342] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6363worker2 # [ 168.212579] nix-daemon[862]: reaped child process 1828, status = succeeded6364worker2 # [ 168.223264] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6365client # [ 168.263443] nix[1416]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6366client: must succeed: systemctl kill -s INT intr6367worker2 # [ 169.325134] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:26Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6368client # [ 169.363199] systemd[1]: intr.service: Sent signal SIGINT to main process 1416 (nix) on client request.6369client # [ 169.373261] nix[1416]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6370client # [ 169.385368] nix[1416]: error: interrupted by the user6371client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6372client # [ 169.409386] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6373client # [ 169.414055] systemd[1]: intr.service: Failed with result 'exit-code'.6374client # [ 169.416675] systemd[1]: intr.service: Consumed 172ms CPU time over 2.519s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6375worker2 # [ 169.469327] nix-daemon[862]: reaped child process 1856, status = succeeded6376worker2 # [ 169.473802] nix-daemon[1858]: unexpected Nix daemon error: error: interrupted by the user6377worker2 # [ 169.488243] nix-daemon[862]: reaped child process 1858, status = failed with exit code 16378(finished: subtest: interrupting the client stops the build on the worker, in 2.67 seconds)6379subtest: a deploy mid-build drains: the build finishes, the new generation takes over6380client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6381client # [ 169.593372] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6382client # Running as unit: sw.service; invocation ID: 3b99f1714ddb4697aa9fa1563d0bbd1b6383client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)6384worker2 # [ 169.750322] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537186385client # [ 169.799134] nix[1437]: don't know how to build these paths:6386client # [ 169.801027] nix[1437]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6387worker2 # [ 169.770104] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:27Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06388worker2 # [ 169.776183] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:27Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16389worker1 # [ 169.807689] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344566390worker1 # [ 169.826286] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344566391client # [ 169.895206] nix[1437]: copying 1 paths to 'lb:50051'...6392worker1 # [ 169.844090] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344566393client # [ 169.898220] nix[1437]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6394worker1 # [ 169.859487] nix-daemon[833]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6395worker1 # [ 170.027205] nix-daemon[833]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)6396lb # [ 170.097930] envoy[913]: conn peer=[2001:db8:1::1]:57536 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3095 tx=1644 ms=98676397lb # [ 170.103231] envoy[913]: conn peer=[2001:db8:1::1]:37940 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=828 ms=23236398worker1 # [ 170.048783] nix-daemon[833]: reaped child process 1606, status = succeeded6399worker1 # [ 170.156607] nix-daemon[833]: accepted connection from pid 1608, user nix-grpc-daemon (trusted)6400worker1 # [ 170.177583] nix-daemon[833]: reaped child process 1611, status = succeeded6401lb # [ 170.246887] niks3-server[1185]: 2026/09/22 13:48:27 INFO Received uploads request method=POST path=/api/pending_closures6402worker1 # [ 170.190403] nix-grpc-daemon[1510]: time=2026-09-22T13:48:27.801Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6403worker1 # [ 170.191925] nix-grpc-daemon[1510]: time=2026-09-22T13:48:27.802Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6404lb # [ 170.280260] niks3-server[1185]: 2026/09/22 13:48:27 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6405lb # [ 170.301229] niks3-server[1185]: 2026/09/22 13:48:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6406worker1 # [ 170.241562] nix-grpc-daemon[1510]: time=2026-09-22T13:48:27.853Z level=INFO msg="Uploading 1 narinfos"6407lb # [ 170.314773] niks3-server[1185]: 2026/09/22 13:48:27 INFO Signed narinfos id=54 count=16408lb # [ 170.321293] niks3-server[1185]: 2026/09/22 13:48:27 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6409lb # [ 170.339815] niks3-server[1185]: 2026/09/22 13:48:27 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6410worker1 # [ 170.281108] nix-grpc-daemon[1510]: time=2026-09-22T13:48:27.893Z level=INFO msg="Upload complete. (404ms)"6411lb # [ 170.350095] niks3-server[1185]: 2026/09/22 13:48:27 INFO Completed upload id=546412lb # [ 170.357296] niks3-server[1185]: 2026/09/22 13:48:27 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6413worker1 # [ 170.292311] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34456 duration_s=0 paths=1 nar_bytes_in=4166414worker1 # [ 170.306257] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344566415worker1 # [ 170.321073] nix-daemon[833]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6416worker1 # [ 170.327813] nix-daemon[833]: reaped child process 1601, status = succeeded6417client # [ 170.389809] nix[1437]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6418worker1 # [ 170.339649] nix-daemon[833]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6419worker1: must succeed: systemctl reload nix-grpc-daemon.service6420worker1 # [ 172.166623] systemd[1]: Reloading Nix worker-protocol over gRPC...6421worker1 # [ 172.216765] systemd[1]: Reloaded Nix worker-protocol over gRPC.6422worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds)6423worker1: must succeed: pgrep -f 'read -t [9]0 x'6424worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6425client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26426worker2 # [ 172.503549] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537186427client # don't know how to build these paths:6428client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6429worker1 # [ 172.502433] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:30Z level=info event=draining6430worker2 # [ 172.513309] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537186431worker2 # [ 172.544926] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06432worker2 # [ 172.551753] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16433worker2 # [ 172.593130] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537186434worker2 # [ 172.615580] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537186435client # copying 1 paths to 'lb:50051'...6436worker2 # [ 172.633165] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537186437client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6438worker2 # [ 172.656476] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6439worker2 # [ 172.829030] nix-daemon[862]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6440worker2 # [ 172.847102] nix-daemon[862]: reaped child process 1908, status = succeeded6441worker2 # [ 172.951779] nix-daemon[862]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6442worker2 # [ 172.972603] nix-daemon[862]: reaped child process 1913, status = succeeded6443lb # [ 173.033259] niks3-server[1185]: 2026/09/22 13:48:30 INFO Received uploads request method=POST path=/api/pending_closures6444worker2 # [ 172.990319] nix-grpc-daemon[1830]: time=2026-09-22T13:48:30.588Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6445worker2 # [ 172.991826] nix-grpc-daemon[1830]: time=2026-09-22T13:48:30.588Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6446lb # [ 173.068825] niks3-server[1185]: 2026/09/22 13:48:30 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6447worker2 # [ 173.030988] nix-grpc-daemon[1830]: time=2026-09-22T13:48:30.629Z level=INFO msg="Uploading 1 narinfos"6448lb # [ 173.087212] niks3-server[1185]: 2026/09/22 13:48:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6449lb # [ 173.096320] niks3-server[1185]: 2026/09/22 13:48:30 INFO Signed narinfos id=55 count=16450lb # [ 173.102452] niks3-server[1185]: 2026/09/22 13:48:30 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6451lb # [ 173.113308] niks3-server[1185]: 2026/09/22 13:48:30 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6452lb # [ 173.122322] niks3-server[1185]: 2026/09/22 13:48:30 INFO Completed upload id=556453worker2 # [ 173.065093] nix-grpc-daemon[1830]: time=2026-09-22T13:48:30.663Z level=INFO msg="Upload complete. (391ms)"6454lb # [ 173.127184] niks3-server[1185]: 2026/09/22 13:48:30 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6455worker2 # [ 173.076323] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=0 paths=1 nar_bytes_in=5286456worker2 # [ 173.092373] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537186457worker2 # [ 173.098647] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6458worker2 # [ 173.103774] nix-daemon[862]: reaped child process 1903, status = succeeded6459client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6460worker2 # [ 173.116893] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6461client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6462client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6463client # farm-a> LOG-farm-a-during-drain6464worker2 # [ 174.225419] nix-daemon[862]: reaped child process 1917, status = succeeded6465worker2 # [ 174.296533] nix-daemon[862]: accepted connection from pid 1933, user nix-grpc-daemon (trusted)6466worker2 # [ 174.319380] nix-daemon[862]: reaped child process 1936, status = succeeded6467lb # [ 174.381240] niks3-server[1185]: 2026/09/22 13:48:31 INFO Received uploads request method=POST path=/api/pending_closures6468worker2 # [ 174.336428] nix-grpc-daemon[1830]: time=2026-09-22T13:48:31.931Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6469worker2 # [ 174.338176] nix-grpc-daemon[1830]: time=2026-09-22T13:48:31.931Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6470lb # [ 174.407860] niks3-server[1185]: 2026/09/22 13:48:32 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6471lb # [ 174.420506] niks3-server[1185]: 2026/09/22 13:48:32 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6472lb # [ 174.430064] niks3-server[1185]: 2026/09/22 13:48:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6473worker2 # [ 174.378486] nix-grpc-daemon[1830]: time=2026-09-22T13:48:31.975Z level=INFO msg="Uploading 1 narinfos"6474lb # [ 174.435965] niks3-server[1185]: 2026/09/22 13:48:32 INFO Signed narinfos id=56 count=16475lb # [ 174.440035] niks3-server[1185]: 2026/09/22 13:48:32 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6476lb # [ 174.446125] niks3-server[1185]: 2026/09/22 13:48:32 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6477worker2 # [ 174.394919] nix-grpc-daemon[1830]: time=2026-09-22T13:48:31.993Z level=INFO msg="Upload complete. (268ms)"6478lb # [ 174.452446] niks3-server[1185]: 2026/09/22 13:48:32 INFO Completed upload id=566479lb # [ 174.454440] niks3-server[1185]: 2026/09/22 13:48:32 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6480worker2 # [ 174.406978] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=16481worker2 # [ 174.424501] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:32Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06482worker2 # [ 174.427773] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:32Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16483worker2 # [ 174.432667] nix-daemon[862]: reaped child process 1915, status = succeeded6484worker2 # [ 174.460722] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537186485worker2 # [ 174.475252] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537186486worker2 # [ 174.490845] nix-grpc-daemon[1830]: time=2026-09-22T13:48:32.089Z level=INFO msg="All 1 paths already cached"6487client # copying 1 paths to 'lb:50051'...6488client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6489worker2 # [ 174.502031] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537186490worker2 # [ 174.516334] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6491worker2 # [ 174.666048] nix-daemon[862]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6492worker2 # [ 174.685337] nix-daemon[862]: reaped child process 1945, status = succeeded6493worker2 # [ 174.790025] nix-daemon[862]: accepted connection from pid 1947, user nix-grpc-daemon (trusted)6494worker2 # [ 174.811190] nix-daemon[862]: reaped child process 1950, status = succeeded6495lb # [ 174.871152] niks3-server[1185]: 2026/09/22 13:48:32 INFO Received uploads request method=POST path=/api/pending_closures6496worker2 # [ 174.825607] nix-grpc-daemon[1830]: time=2026-09-22T13:48:32.423Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6497worker2 # [ 174.827273] nix-grpc-daemon[1830]: time=2026-09-22T13:48:32.423Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6498lb # [ 174.907828] niks3-server[1185]: 2026/09/22 13:48:32 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6499worker2 # [ 174.867325] nix-grpc-daemon[1830]: time=2026-09-22T13:48:32.465Z level=INFO msg="Uploading 1 narinfos"6500lb # [ 174.923646] niks3-server[1185]: 2026/09/22 13:48:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6501lb # [ 174.934190] niks3-server[1185]: 2026/09/22 13:48:32 INFO Signed narinfos id=57 count=16502lb # [ 174.941238] niks3-server[1185]: 2026/09/22 13:48:32 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6503worker2 # [ 174.898625] nix-grpc-daemon[1830]: time=2026-09-22T13:48:32.497Z level=INFO msg="Upload complete. (367ms)"6504lb # [ 174.954549] niks3-server[1185]: 2026/09/22 13:48:32 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6505lb # [ 174.964689] niks3-server[1185]: 2026/09/22 13:48:32 INFO Completed upload id=576506worker2 # [ 174.908708] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=0 paths=1 nar_bytes_in=6406507lb # [ 174.967488] niks3-server[1185]: 2026/09/22 13:48:32 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6508worker2 # [ 174.922488] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537186509worker2 # [ 174.928524] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6510worker2 # [ 174.933447] nix-daemon[862]: reaped child process 1940, status = succeeded6511client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6512worker2 # [ 174.945440] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6513client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6514client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6515client # farm-b> LOG-farm-b-during-drain6516worker2 # [ 176.611454] nix-daemon[862]: reaped child process 1954, status = succeeded6517worker2 # [ 176.729154] nix-daemon[862]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)6518worker2 # [ 176.749140] nix-daemon[862]: reaped child process 1973, status = succeeded6519lb # [ 176.810947] niks3-server[1185]: 2026/09/22 13:48:34 INFO Received uploads request method=POST path=/api/pending_closures6520worker2 # [ 176.766379] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.363Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6521worker2 # [ 176.768562] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.363Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6522lb # [ 176.850933] niks3-server[1185]: 2026/09/22 13:48:34 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6523worker2 # [ 176.812884] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.410Z level=INFO msg="Uploading 1 narinfos"6524lb # [ 176.867841] niks3-server[1185]: 2026/09/22 13:48:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6525lb # [ 176.877732] niks3-server[1185]: 2026/09/22 13:48:34 INFO Signed narinfos id=58 count=16526lb # [ 176.885586] niks3-server[1185]: 2026/09/22 13:48:34 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6527lb # [ 176.894073] niks3-server[1185]: 2026/09/22 13:48:34 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6528lb # [ 176.904475] niks3-server[1185]: 2026/09/22 13:48:34 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6529lb # [ 176.909721] niks3-server[1185]: 2026/09/22 13:48:34 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6530lb # [ 176.913646] niks3-server[1185]: 2026/09/22 13:48:34 INFO Completed upload id=586531worker2 # [ 176.857813] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.456Z level=INFO msg="Upload complete. (302ms)"6532worker2 # [ 176.876498] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=16533worker2 # [ 176.892401] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:34Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06534worker2 # [ 176.899231] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:34Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16535worker2 # [ 176.903697] nix-daemon[862]: reaped child process 1952, status = succeeded6536worker2 # [ 176.921338] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537186537worker2 # [ 176.936928] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537186538worker2 # [ 176.951915] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.550Z level=INFO msg="All 2 paths already cached"6539client # copying 1 paths to 'lb:50051'...6540worker2 # [ 176.963639] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537186541client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6542worker2 # [ 176.986498] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6543worker2 # [ 177.152594] nix-daemon[862]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)6544worker2 # [ 177.174080] nix-daemon[862]: reaped child process 1982, status = succeeded6545worker2 # [ 177.277103] nix-daemon[862]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)6546worker2 # [ 177.294027] nix-daemon[862]: reaped child process 1987, status = succeeded6547lb # [ 177.356797] niks3-server[1185]: 2026/09/22 13:48:34 INFO Received uploads request method=POST path=/api/pending_closures6548worker2 # [ 177.310230] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.908Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6549worker2 # [ 177.311780] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.908Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6550lb # [ 177.384293] niks3-server[1185]: 2026/09/22 13:48:34 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6551worker2 # [ 177.344219] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.942Z level=INFO msg="Uploading 1 narinfos"6552lb # [ 177.399975] niks3-server[1185]: 2026/09/22 13:48:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6553lb # [ 177.407508] niks3-server[1185]: 2026/09/22 13:48:35 INFO Signed narinfos id=59 count=16554lb # [ 177.416075] niks3-server[1185]: 2026/09/22 13:48:35 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6555lb # [ 177.423735] niks3-server[1185]: 2026/09/22 13:48:35 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6556worker2 # [ 177.374274] nix-grpc-daemon[1830]: time=2026-09-22T13:48:34.972Z level=INFO msg="Upload complete. (369ms)"6557lb # [ 177.433015] niks3-server[1185]: 2026/09/22 13:48:35 INFO Completed upload id=596558lb # [ 177.436410] niks3-server[1185]: 2026/09/22 13:48:35 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6559worker2 # [ 177.384374] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=0 paths=1 nar_bytes_in=7686560worker2 # [ 177.395799] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537186561worker2 # [ 177.404527] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6562worker2 # [ 177.407110] nix-daemon[862]: reaped child process 1977, status = succeeded6563client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6564worker2 # [ 177.417519] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6565client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6566client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6567client # farm-top> LOG-farm-top-during-drain6568worker2 # [ 178.469223] nix-daemon[862]: reaped child process 1991, status = succeeded6569worker2 # [ 178.580005] nix-daemon[862]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)6570worker2 # [ 178.603016] nix-daemon[862]: reaped child process 2010, status = succeeded6571lb # [ 178.657790] niks3-server[1185]: 2026/09/22 13:48:36 INFO Received uploads request method=POST path=/api/pending_closures6572worker2 # [ 178.612548] nix-grpc-daemon[1830]: time=2026-09-22T13:48:36.210Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6573worker2 # [ 178.614022] nix-grpc-daemon[1830]: time=2026-09-22T13:48:36.210Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6574lb # [ 178.703045] niks3-server[1185]: 2026/09/22 13:48:36 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6575lb # [ 178.714939] niks3-server[1185]: 2026/09/22 13:48:36 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6576worker2 # [ 178.666483] nix-grpc-daemon[1830]: time=2026-09-22T13:48:36.264Z level=INFO msg="Uploading 1 narinfos"6577lb # [ 178.724834] niks3-server[1185]: 2026/09/22 13:48:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6578lb # [ 178.733954] niks3-server[1185]: 2026/09/22 13:48:36 INFO Signed narinfos id=60 count=16579lb # [ 178.739890] niks3-server[1185]: 2026/09/22 13:48:36 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6580worker2 # [ 178.689561] nix-grpc-daemon[1830]: time=2026-09-22T13:48:36.288Z level=INFO msg="Upload complete. (259ms)"6581lb # [ 178.746198] niks3-server[1185]: 2026/09/22 13:48:36 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6582lb # [ 178.749774] niks3-server[1185]: 2026/09/22 13:48:36 INFO Completed upload id=606583lb # [ 178.751570] niks3-server[1185]: 2026/09/22 13:48:36 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6584worker2 # [ 178.704623] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=16585worker2 # [ 178.720463] nix-daemon[862]: reaped child process 1989, status = succeeded6586client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.47 seconds)6587client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6588client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.17 seconds)6589worker1: must succeed: pkill -f 'read -t [9]0 x'6590worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6591client: waiting for success: ! systemctl is-active sw6592worker1 # [ 179.219359] nix-daemon[833]: reaped child process 1615, status = succeeded6593worker1 # [ 179.334143] nix-daemon[833]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6594worker1 # [ 179.353411] nix-daemon[833]: reaped child process 1659, status = succeeded6595lb # [ 179.424303] niks3-server[1185]: 2026/09/22 13:48:37 INFO Received uploads request method=POST path=/api/pending_closures6596worker1 # [ 179.374192] nix-grpc-daemon[1510]: time=2026-09-22T13:48:36.985Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6597worker1 # [ 179.375958] nix-grpc-daemon[1510]: time=2026-09-22T13:48:36.985Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6598lb # [ 179.470680] niks3-server[1185]: 2026/09/22 13:48:37 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6599lb # [ 179.482405] niks3-server[1185]: 2026/09/22 13:48:37 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6600worker1 # [ 179.425662] nix-grpc-daemon[1510]: time=2026-09-22T13:48:37.037Z level=INFO msg="Uploading 1 narinfos"6601lb # [ 179.491665] niks3-server[1185]: 2026/09/22 13:48:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6602lb # [ 179.502669] niks3-server[1185]: 2026/09/22 13:48:37 INFO Signed narinfos id=61 count=16603lb # [ 179.510784] niks3-server[1185]: 2026/09/22 13:48:37 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6604lb # [ 179.522222] niks3-server[1185]: 2026/09/22 13:48:37 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6605worker1 # [ 179.460357] nix-grpc-daemon[1510]: time=2026-09-22T13:48:37.072Z level=INFO msg="Upload complete. (298ms)"6606lb # [ 179.530318] niks3-server[1185]: 2026/09/22 13:48:37 INFO Completed upload id=616607lb # [ 179.534592] niks3-server[1185]: 2026/09/22 13:48:37 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6608worker1 # [ 179.473086] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34456 duration_s=9 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=16609worker1 # [ 179.494162] nix-daemon[833]: reaped child process 1613, status = succeeded6610client # [ 179.548311] systemd[1]: sw.service: Deactivated successfully.6611client # [ 179.551091] systemd[1]: sw.service: Consumed 183ms CPU time over 9.956s wall clock time, 39.2M memory peak, 7.5K incoming IP traffic, 8.7K outgoing IP traffic.6612worker1 # [ 179.504531] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:37Z level=info event=signal_exit6613lb # [ 180.100167] envoy[913]: conn peer=[2001:db8:1::1]:35740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9638 tx=4443 ms=62146614lb # [ 180.100497] envoy[913]: conn peer=[2001:db8:1::1]:37946 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=1534 ms=97476615client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6616client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6617client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6618worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6619worker2 # [ 181.522054] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400426620worker1 # [ 181.516842] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:39Z level=info event=scheduler_restarting6621worker2 # [ 181.527345] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:39Z level=info event=worker_gone addr=192.168.1.3:500516622worker1 # [ 181.536694] nix-grpc-daemon[1390]: ts=2026-09-22T13:48:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6623worker1 # [ 181.562205] nix-daemon[833]: reaped child process 1449, status = succeeded6624worker1 # [ 181.814842] nix-grpc-daemon[1390]: E0000 00:00:1790084919.426849 1435 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6625worker1 # [ 181.822079] nix-grpc-daemon[1390]: E0000 00:00:1790084919.434131 1435 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6626worker1 # [ 181.917438] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6627worker1 # [ 181.924477] systemd[1]: nix-grpc-daemon.service: Consumed 1.268s CPU time over 40.588s wall clock time, 36M memory peak, 26.6K incoming IP traffic, 38.9K outgoing IP traffic.6628worker1 # [ 181.944852] systemd[1]: Starting Nix worker-protocol over gRPC...6629worker1 # [ 182.095750] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6630worker1 # [ 182.098289] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6631worker1 # [ 182.102124] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16632worker1 # [ 182.108078] nix-grpc-daemon[1673]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6633worker1 # [ 182.109628] nix-grpc-daemon[1673]: E0000 00:00:1790084919.719047 1673 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6634worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.05 seconds)6635lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6636lb: must succeed: curl -sf localhost:9901/clusters6637worker1 # [ 182.156451] nix-grpc-daemon[1673]: E0000 00:00:1790084919.725569 1673 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6638worker1 # [ 182.159405] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6639worker1 # [ 182.161422] nix-grpc-daemon[1673]: E0000 00:00:1790084919.732123 1712 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6640worker1 # [ 182.163760] nix-grpc-daemon[1673]: E0000 00:00:1790084919.750194 1712 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6641worker1 # [ 182.166214] nix-grpc-daemon[1673]: E0000 00:00:1790084919.750374 1712 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6642worker1 # [ 182.168589] nix-grpc-daemon[1673]: E0000 00:00:1790084919.766870 1712 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6643worker1 # [ 182.174165] systemd[1]: Started Nix worker-protocol over gRPC.6644worker1 # [ 182.176272] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:39Z level=info event=scheduler_connected addr=lb:500516645worker2 # [ 182.189453] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:400426646worker2 # [ 182.189856] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06647lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6648lb: must succeed: curl -sf localhost:9901/clusters6649lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6650lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.16 seconds)6651lb: must succeed: curl -sf localhost:9901/clusters6652lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6653worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6654worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6655(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.96 seconds)6656subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6657client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16658worker2 # [ 183.678582] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537186659worker1 # [ 183.677235] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526660worker1 # [ 183.692470] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6661worker1 # [ 183.717221] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)6662worker1 # [ 183.783945] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 2/5)6663worker1 # [ 183.819604] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 3/5)6664worker1 # [ 184.130917] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 4/5)6665worker1 # [ 184.285695] nix-grpc-daemon[1673]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6666worker1 # [ 184.293600] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6667worker2 # [ 184.362152] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:41Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06668worker2 # [ 184.362539] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:41Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16669worker2 # [ 184.362663] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:41Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06670worker2 # [ 184.362778] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:41Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06671worker1 # [ 184.399307] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526672worker1 # [ 184.418280] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526673worker1 # [ 184.434960] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392526674worker1 # [ 184.449969] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6675worker1 # [ 184.484099] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:42Z level=info event=niks3_push_started pid=17336676worker1 # [ 184.633382] nix-grpc-daemon[1733]: time=2026-09-22T13:48:42.244Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6677worker1 # [ 184.746746] nix-daemon[833]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6678worker1 # [ 184.770297] nix-daemon[833]: reaped child process 1749, status = succeeded6679worker1 # [ 184.874423] nix-daemon[833]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)6680worker1 # [ 184.895922] nix-daemon[833]: reaped child process 1754, status = succeeded6681lb # [ 184.965284] niks3-server[1185]: 2026/09/22 13:48:42 INFO Received uploads request method=POST path=/api/pending_closures6682worker1 # [ 184.910767] nix-grpc-daemon[1733]: time=2026-09-22T13:48:42.521Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6683worker1 # [ 184.912724] nix-grpc-daemon[1733]: time=2026-09-22T13:48:42.521Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6684lb # [ 185.007120] niks3-server[1185]: 2026/09/22 13:48:42 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6685worker1 # [ 184.961928] nix-grpc-daemon[1733]: time=2026-09-22T13:48:42.573Z level=INFO msg="Uploading 1 narinfos"6686lb # [ 185.027682] niks3-server[1185]: 2026/09/22 13:48:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6687lb # [ 185.040407] niks3-server[1185]: 2026/09/22 13:48:42 INFO Signed narinfos id=62 count=16688lb # [ 185.043802] niks3-server[1185]: 2026/09/22 13:48:42 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6689lb # [ 185.065418] niks3-server[1185]: 2026/09/22 13:48:42 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6690worker1 # [ 185.008172] nix-grpc-daemon[1733]: time=2026-09-22T13:48:42.620Z level=INFO msg="Upload complete. (374ms)"6691lb # [ 185.076436] niks3-server[1185]: 2026/09/22 13:48:42 INFO Completed upload id=626692lb # [ 185.080305] niks3-server[1185]: 2026/09/22 13:48:42 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6693worker1 # [ 185.021520] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=5126694worker1 # [ 185.037574] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526695worker1 # [ 185.048261] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6696worker1 # [ 185.051163] nix-daemon[833]: reaped child process 1731, status = succeeded6697worker1 # [ 185.062973] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6698worker1 # [ 186.177351] nix-daemon[833]: reaped child process 1758, status = succeeded6699worker1 # [ 186.247999] nix-daemon[833]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6700worker1 # [ 186.271407] nix-daemon[833]: reaped child process 1777, status = succeeded6701lb # [ 186.342965] niks3-server[1185]: 2026/09/22 13:48:43 INFO Received uploads request method=POST path=/api/pending_closures6702worker1 # [ 186.291592] nix-grpc-daemon[1733]: time=2026-09-22T13:48:43.900Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6703worker1 # [ 186.295242] nix-grpc-daemon[1733]: time=2026-09-22T13:48:43.900Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6704lb # [ 186.385348] niks3-server[1185]: 2026/09/22 13:48:43 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6705lb # [ 186.407109] niks3-server[1185]: 2026/09/22 13:48:44 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6706lb # [ 186.422287] niks3-server[1185]: 2026/09/22 13:48:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6707worker1 # [ 186.356123] nix-grpc-daemon[1733]: time=2026-09-22T13:48:43.967Z level=INFO msg="Uploading 1 narinfos"6708lb # [ 186.430689] niks3-server[1185]: 2026/09/22 13:48:44 INFO Signed narinfos id=63 count=16709lb # [ 186.437591] niks3-server[1185]: 2026/09/22 13:48:44 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6710lb # [ 186.449471] niks3-server[1185]: 2026/09/22 13:48:44 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6711lb # [ 186.457325] niks3-server[1185]: 2026/09/22 13:48:44 INFO Completed upload id=636712worker1 # [ 186.390075] nix-grpc-daemon[1733]: time=2026-09-22T13:48:44.002Z level=INFO msg="Upload complete. (313ms)"6713lb # [ 186.461901] niks3-server[1185]: 2026/09/22 13:48:44 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6714worker2 # [ 186.414337] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:44Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16715worker2 # [ 186.427703] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:44Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06716worker1 # [ 186.405315] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=16717worker1 # [ 186.426750] nix-daemon[833]: reaped child process 1756, status = succeeded6718worker1 # [ 186.459967] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526719worker1 # [ 186.477022] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526720worker1 # [ 186.494356] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392526721worker1 # [ 186.509617] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6722worker1 # [ 186.678093] nix-daemon[833]: accepted connection from pid 1781, user nix-grpc-daemon (trusted)6723worker1 # [ 186.703922] nix-daemon[833]: reaped child process 1784, status = succeeded6724worker1 # [ 186.816529] nix-daemon[833]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6725worker1 # [ 186.838063] nix-daemon[833]: reaped child process 1789, status = succeeded6726lb # [ 186.902918] niks3-server[1185]: 2026/09/22 13:48:44 INFO Received uploads request method=POST path=/api/pending_closures6727worker1 # [ 186.846754] nix-grpc-daemon[1733]: time=2026-09-22T13:48:44.458Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6728worker1 # [ 186.848467] nix-grpc-daemon[1733]: time=2026-09-22T13:48:44.458Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6729lb # [ 186.937471] niks3-server[1185]: 2026/09/22 13:48:44 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6730worker1 # [ 186.890841] nix-grpc-daemon[1733]: time=2026-09-22T13:48:44.502Z level=INFO msg="Uploading 1 narinfos"6731lb # [ 186.956922] niks3-server[1185]: 2026/09/22 13:48:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6732lb # [ 186.964780] niks3-server[1185]: 2026/09/22 13:48:44 INFO Signed narinfos id=64 count=16733lb # [ 186.973425] niks3-server[1185]: 2026/09/22 13:48:44 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6734lb # [ 186.990409] niks3-server[1185]: 2026/09/22 13:48:44 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6735lb # [ 186.999004] niks3-server[1185]: 2026/09/22 13:48:44 INFO Completed upload id=646736worker1 # [ 186.931538] nix-grpc-daemon[1733]: time=2026-09-22T13:48:44.543Z level=INFO msg="Upload complete. (404ms)"6737lb # [ 187.002250] niks3-server[1185]: 2026/09/22 13:48:44 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6738worker1 # [ 186.945001] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=5126739worker1 # [ 186.964394] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526740worker1 # [ 186.969837] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6741worker1 # [ 186.976477] nix-daemon[833]: reaped child process 1779, status = succeeded6742worker1 # [ 186.988402] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6743worker1 # [ 188.080768] nix-daemon[833]: reaped child process 1793, status = succeeded6744worker1 # [ 188.180138] nix-daemon[833]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6745worker1 # [ 188.198127] nix-daemon[833]: reaped child process 1812, status = succeeded6746lb # [ 188.274157] niks3-server[1185]: 2026/09/22 13:48:45 INFO Received uploads request method=POST path=/api/pending_closures6747worker1 # [ 188.219413] nix-grpc-daemon[1733]: time=2026-09-22T13:48:45.830Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6748worker1 # [ 188.221295] nix-grpc-daemon[1733]: time=2026-09-22T13:48:45.830Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6749lb # [ 188.317363] niks3-server[1185]: 2026/09/22 13:48:45 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6750lb # [ 188.330773] niks3-server[1185]: 2026/09/22 13:48:45 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6751lb # [ 188.341524] niks3-server[1185]: 2026/09/22 13:48:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6752worker1 # [ 188.275663] nix-grpc-daemon[1733]: time=2026-09-22T13:48:45.887Z level=INFO msg="Uploading 1 narinfos"6753lb # [ 188.349758] niks3-server[1185]: 2026/09/22 13:48:45 INFO Signed narinfos id=65 count=16754lb # [ 188.355256] niks3-server[1185]: 2026/09/22 13:48:45 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6755lb # [ 188.367015] niks3-server[1185]: 2026/09/22 13:48:45 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6756lb # [ 188.374518] niks3-server[1185]: 2026/09/22 13:48:45 INFO Completed upload id=656757lb # [ 188.378202] niks3-server[1185]: 2026/09/22 13:48:45 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6758worker1 # [ 188.307595] nix-grpc-daemon[1733]: time=2026-09-22T13:48:45.919Z level=INFO msg="Upload complete. (302ms)"6759worker2 # [ 188.334602] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:45Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16760worker1 # [ 188.322911] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=16761worker2 # [ 188.343059] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:45Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06762worker1 # [ 188.342632] nix-daemon[833]: reaped child process 1791, status = succeeded6763worker1 # [ 188.366864] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526764worker1 # [ 188.383615] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526765worker1 # [ 188.399266] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392526766worker1 # [ 188.413799] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6767worker1 # [ 188.566968] nix-daemon[833]: accepted connection from pid 1816, user nix-grpc-daemon (trusted)6768worker1 # [ 188.587754] nix-daemon[833]: reaped child process 1819, status = succeeded6769worker1 # [ 188.691965] nix-daemon[833]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6770worker1 # [ 188.711781] nix-daemon[833]: reaped child process 1824, status = succeeded6771lb # [ 188.782404] niks3-server[1185]: 2026/09/22 13:48:46 INFO Received uploads request method=POST path=/api/pending_closures6772worker1 # [ 188.727224] nix-grpc-daemon[1733]: time=2026-09-22T13:48:46.339Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6773worker1 # [ 188.728945] nix-grpc-daemon[1733]: time=2026-09-22T13:48:46.339Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6774lb # [ 188.815937] niks3-server[1185]: 2026/09/22 13:48:46 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6775worker1 # [ 188.764825] nix-grpc-daemon[1733]: time=2026-09-22T13:48:46.377Z level=INFO msg="Uploading 1 narinfos"6776lb # [ 188.831027] niks3-server[1185]: 2026/09/22 13:48:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6777lb # [ 188.839711] niks3-server[1185]: 2026/09/22 13:48:46 INFO Signed narinfos id=66 count=16778lb # [ 188.847599] niks3-server[1185]: 2026/09/22 13:48:46 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6779lb # [ 188.862096] niks3-server[1185]: 2026/09/22 13:48:46 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6780worker1 # [ 188.800820] nix-grpc-daemon[1733]: time=2026-09-22T13:48:46.413Z level=INFO msg="Upload complete. (371ms)"6781lb # [ 188.867582] niks3-server[1185]: 2026/09/22 13:48:46 INFO Completed upload id=666782lb # [ 188.873024] niks3-server[1185]: 2026/09/22 13:48:46 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6783worker1 # [ 188.808121] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=5126784worker1 # [ 188.823456] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526785worker1 # [ 188.831108] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6786worker1 # [ 188.837178] nix-daemon[833]: reaped child process 1814, status = succeeded6787worker1 # [ 188.851650] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6788lb # [ 190.098950] envoy[913]: conn peer=[2001:db8:1::3]:37392 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2445 tx=766 ms=399756789worker1 # [ 190.329236] nix-daemon[833]: reaped child process 1828, status = succeeded6790worker1 # [ 190.434611] nix-daemon[833]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6791worker1 # [ 190.456390] nix-daemon[833]: reaped child process 1847, status = succeeded6792lb # [ 190.527118] niks3-server[1185]: 2026/09/22 13:48:48 INFO Received uploads request method=POST path=/api/pending_closures6793worker1 # [ 190.471536] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.082Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6794worker1 # [ 190.473548] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.082Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6795lb # [ 190.567793] niks3-server[1185]: 2026/09/22 13:48:48 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6796lb # [ 190.581386] niks3-server[1185]: 2026/09/22 13:48:48 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6797worker1 # [ 190.531535] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.143Z level=INFO msg="Uploading 1 narinfos"6798lb # [ 190.597701] niks3-server[1185]: 2026/09/22 13:48:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6799lb # [ 190.607120] niks3-server[1185]: 2026/09/22 13:48:48 INFO Signed narinfos id=67 count=16800lb # [ 190.613951] niks3-server[1185]: 2026/09/22 13:48:48 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6801lb # [ 190.625634] niks3-server[1185]: 2026/09/22 13:48:48 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6802worker1 # [ 190.566653] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.178Z level=INFO msg="Upload complete. (303ms)"6803lb # [ 190.636260] niks3-server[1185]: 2026/09/22 13:48:48 INFO Completed upload id=676804lb # [ 190.638335] niks3-server[1185]: 2026/09/22 13:48:48 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6805worker1 # [ 190.577119] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=16806worker2 # [ 190.591704] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:48Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16807worker2 # [ 190.602702] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:48Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06808worker1 # [ 190.597097] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526809worker1 # [ 190.611855] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526810worker1 # [ 190.618083] nix-daemon[833]: reaped child process 1826, status = succeeded6811worker1 # [ 190.625199] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.237Z level=INFO msg="All 1 paths already cached"6812worker1 # [ 190.634039] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392526813worker1 # [ 190.650154] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6814worker1 # [ 190.792107] nix-daemon[833]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6815worker1 # [ 190.811299] nix-daemon[833]: reaped child process 1854, status = succeeded6816worker1 # [ 190.911937] nix-daemon[833]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)6817worker1 # [ 190.930395] nix-daemon[833]: reaped child process 1859, status = succeeded6818lb # [ 191.001197] niks3-server[1185]: 2026/09/22 13:48:48 INFO Received uploads request method=POST path=/api/pending_closures6819worker1 # [ 190.943723] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.555Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6820worker1 # [ 190.945452] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.555Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6821lb # [ 191.031483] niks3-server[1185]: 2026/09/22 13:48:48 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6822worker1 # [ 190.978778] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.591Z level=INFO msg="Uploading 1 narinfos"6823lb # [ 191.046048] niks3-server[1185]: 2026/09/22 13:48:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6824lb # [ 191.058556] niks3-server[1185]: 2026/09/22 13:48:48 INFO Signed narinfos id=68 count=16825lb # [ 191.066680] niks3-server[1185]: 2026/09/22 13:48:48 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6826lb # [ 191.081202] niks3-server[1185]: 2026/09/22 13:48:48 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6827lb # [ 191.091609] niks3-server[1185]: 2026/09/22 13:48:48 INFO Completed upload id=686828worker1 # [ 191.022019] nix-grpc-daemon[1733]: time=2026-09-22T13:48:48.634Z level=INFO msg="Upload complete. (361ms)"6829lb # [ 191.095915] niks3-server[1185]: 2026/09/22 13:48:48 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6830worker1 # [ 191.032565] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=6246831worker1 # [ 191.048720] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526832worker1 # [ 191.056374] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6833worker1 # [ 191.062274] nix-daemon[833]: reaped child process 1849, status = succeeded6834worker1 # [ 191.075822] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6835worker1 # [ 192.178234] nix-daemon[833]: reaped child process 1863, status = succeeded6836worker1 # [ 192.263399] nix-daemon[833]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)6837worker1 # [ 192.286392] nix-daemon[833]: reaped child process 1882, status = succeeded6838lb # [ 192.358662] niks3-server[1185]: 2026/09/22 13:48:49 INFO Received uploads request method=POST path=/api/pending_closures6839worker1 # [ 192.301949] nix-grpc-daemon[1733]: time=2026-09-22T13:48:49.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6840worker1 # [ 192.306806] nix-grpc-daemon[1733]: time=2026-09-22T13:48:49.916Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6841lb # [ 192.404737] niks3-server[1185]: 2026/09/22 13:48:50 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6842worker1 # [ 192.355081] nix-grpc-daemon[1733]: time=2026-09-22T13:48:49.966Z level=INFO msg="Uploading 1 narinfos"6843lb # [ 192.420807] niks3-server[1185]: 2026/09/22 13:48:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6844lb # [ 192.433826] niks3-server[1185]: 2026/09/22 13:48:50 INFO Signed narinfos id=69 count=16845lb # [ 192.441824] niks3-server[1185]: 2026/09/22 13:48:50 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6846lb # [ 192.451771] niks3-server[1185]: 2026/09/22 13:48:50 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6847lb # [ 192.460229] niks3-server[1185]: 2026/09/22 13:48:50 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6848lb # [ 192.467702] niks3-server[1185]: 2026/09/22 13:48:50 INFO Completed upload id=696849worker1 # [ 192.400915] nix-grpc-daemon[1733]: time=2026-09-22T13:48:50.013Z level=INFO msg="Upload complete. (310ms)"6850lb # [ 192.470584] niks3-server[1185]: 2026/09/22 13:48:50 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6851worker2 # [ 192.421479] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:50Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16852worker1 # [ 192.413302] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=16853worker2 # [ 192.434538] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:50Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06854worker1 # [ 192.434947] nix-daemon[833]: reaped child process 1861, status = succeeded6855worker1 # [ 192.465971] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526856worker1 # [ 192.482595] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526857worker1 # [ 192.497457] nix-grpc-daemon[1733]: time=2026-09-22T13:48:50.109Z level=INFO msg="All 1 paths already cached"6858worker1 # [ 192.507206] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392526859worker1 # [ 192.520871] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6860worker1 # [ 192.677277] nix-daemon[833]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6861worker1 # [ 192.699399] nix-daemon[833]: reaped child process 1889, status = succeeded6862worker1 # [ 192.800116] nix-daemon[833]: accepted connection from pid 1891, user nix-grpc-daemon (trusted)6863worker1 # [ 192.819041] nix-daemon[833]: reaped child process 1894, status = succeeded6864lb # [ 192.888247] niks3-server[1185]: 2026/09/22 13:48:50 INFO Received uploads request method=POST path=/api/pending_closures6865worker1 # [ 192.832915] nix-grpc-daemon[1733]: time=2026-09-22T13:48:50.445Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6866worker1 # [ 192.834410] nix-grpc-daemon[1733]: time=2026-09-22T13:48:50.445Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6867lb # [ 192.926991] niks3-server[1185]: 2026/09/22 13:48:50 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6868worker1 # [ 192.876541] nix-grpc-daemon[1733]: time=2026-09-22T13:48:50.488Z level=INFO msg="Uploading 1 narinfos"6869lb # [ 192.943095] niks3-server[1185]: 2026/09/22 13:48:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6870lb # [ 192.949937] niks3-server[1185]: 2026/09/22 13:48:50 INFO Signed narinfos id=70 count=16871lb # [ 192.957013] niks3-server[1185]: 2026/09/22 13:48:50 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6872lb # [ 192.973685] niks3-server[1185]: 2026/09/22 13:48:50 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6873lb # [ 192.982165] niks3-server[1185]: 2026/09/22 13:48:50 INFO Completed upload id=706874lb # [ 192.984566] niks3-server[1185]: 2026/09/22 13:48:50 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6875worker1 # [ 192.913121] nix-grpc-daemon[1733]: time=2026-09-22T13:48:50.525Z level=INFO msg="Upload complete. (374ms)"6876worker1 # [ 192.924599] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=6246877worker1 # [ 192.943064] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526878worker1 # [ 192.953028] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6879worker1 # [ 192.958267] nix-daemon[833]: reaped child process 1884, status = succeeded6880worker1 # [ 192.973986] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6881worker1 # [ 194.421624] nix-daemon[833]: reaped child process 1898, status = succeeded6882worker1 # [ 194.539935] nix-daemon[833]: accepted connection from pid 1914, user nix-grpc-daemon (trusted)6883worker1 # [ 194.560742] nix-daemon[833]: reaped child process 1917, status = succeeded6884lb # [ 194.633811] niks3-server[1185]: 2026/09/22 13:48:52 INFO Received uploads request method=POST path=/api/pending_closures6885worker1 # [ 194.580311] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.189Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6886worker1 # [ 194.581919] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.189Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6887lb # [ 194.675283] niks3-server[1185]: 2026/09/22 13:48:52 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6888lb # [ 194.690352] niks3-server[1185]: 2026/09/22 13:48:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6889worker1 # [ 194.629187] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.241Z level=INFO msg="Uploading 1 narinfos"6890lb # [ 194.699077] niks3-server[1185]: 2026/09/22 13:48:52 INFO Signed narinfos id=71 count=16891lb # [ 194.705038] niks3-server[1185]: 2026/09/22 13:48:52 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6892lb # [ 194.712977] niks3-server[1185]: 2026/09/22 13:48:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6893worker1 # [ 194.655542] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.267Z level=INFO msg="Upload complete. (266ms)"6894lb # [ 194.723185] niks3-server[1185]: 2026/09/22 13:48:52 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6895lb # [ 194.732313] niks3-server[1185]: 2026/09/22 13:48:52 INFO Completed upload id=716896lb # [ 194.735574] niks3-server[1185]: 2026/09/22 13:48:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6897worker2 # [ 194.678961] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:52Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16898worker1 # [ 194.670233] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=16899worker2 # [ 194.687972] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:52Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06900worker1 # [ 194.688310] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526901worker1 # [ 194.694464] nix-daemon[833]: reaped child process 1896, status = succeeded6902worker1 # [ 194.702540] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526903worker1 # [ 194.715749] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.328Z level=INFO msg="All 1 paths already cached"6904worker1 # [ 194.725075] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392526905worker1 # [ 194.738513] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6906worker1 # [ 194.885008] nix-daemon[833]: accepted connection from pid 1921, user nix-grpc-daemon (trusted)6907worker1 # [ 194.906237] nix-daemon[833]: reaped child process 1924, status = succeeded6908worker1 # [ 195.010594] nix-daemon[833]: accepted connection from pid 1926, user nix-grpc-daemon (trusted)6909worker1 # [ 195.027460] nix-daemon[833]: reaped child process 1929, status = succeeded6910lb # [ 195.100855] niks3-server[1185]: 2026/09/22 13:48:52 INFO Received uploads request method=POST path=/api/pending_closures6911worker1 # [ 195.044063] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.656Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6912worker1 # [ 195.045588] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.656Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6913lb # [ 195.141459] niks3-server[1185]: 2026/09/22 13:48:52 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6914worker1 # [ 195.089514] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.701Z level=INFO msg="Uploading 1 narinfos"6915lb # [ 195.156298] niks3-server[1185]: 2026/09/22 13:48:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6916lb # [ 195.164707] niks3-server[1185]: 2026/09/22 13:48:52 INFO Signed narinfos id=72 count=16917lb # [ 195.171574] niks3-server[1185]: 2026/09/22 13:48:52 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6918lb # [ 195.182739] niks3-server[1185]: 2026/09/22 13:48:52 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6919lb # [ 195.191345] niks3-server[1185]: 2026/09/22 13:48:52 INFO Completed upload id=726920worker1 # [ 195.124198] nix-grpc-daemon[1733]: time=2026-09-22T13:48:52.736Z level=INFO msg="Upload complete. (373ms)"6921lb # [ 195.197975] niks3-server[1185]: 2026/09/22 13:48:52 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6922worker1 # [ 195.136340] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=6246923worker1 # [ 195.151148] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526924worker1 # [ 195.158312] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6925worker1 # [ 195.163394] nix-daemon[833]: reaped child process 1919, status = succeeded6926worker1 # [ 195.175607] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6927worker1 # [ 196.640814] nix-daemon[833]: reaped child process 1933, status = succeeded6928worker1 # [ 196.767234] nix-daemon[833]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6929worker1 # [ 196.785010] nix-daemon[833]: reaped child process 1952, status = succeeded6930lb # [ 196.858833] niks3-server[1185]: 2026/09/22 13:48:54 INFO Received uploads request method=POST path=/api/pending_closures6931worker1 # [ 196.804457] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.415Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6932worker1 # [ 196.806208] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.415Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6933lb # [ 196.907804] niks3-server[1185]: 2026/09/22 13:48:54 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6934lb # [ 196.924070] niks3-server[1185]: 2026/09/22 13:48:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6935worker1 # [ 196.862713] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.474Z level=INFO msg="Uploading 1 narinfos"6936lb # [ 196.932777] niks3-server[1185]: 2026/09/22 13:48:54 INFO Signed narinfos id=73 count=16937lb # [ 196.938812] niks3-server[1185]: 2026/09/22 13:48:54 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6938lb # [ 196.948679] niks3-server[1185]: 2026/09/22 13:48:54 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6939worker1 # [ 196.891024] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.503Z level=INFO msg="Upload complete. (301ms)"6940lb # [ 196.958080] niks3-server[1185]: 2026/09/22 13:48:54 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6941lb # [ 196.966774] niks3-server[1185]: 2026/09/22 13:48:54 INFO Completed upload id=736942worker2 # [ 196.911285] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:54Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16943lb # [ 196.970647] niks3-server[1185]: 2026/09/22 13:48:54 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6944worker1 # [ 196.905901] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=16945worker2 # [ 196.924598] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:54Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06946worker1 # [ 196.927839] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526947worker1 # [ 196.935259] nix-daemon[833]: reaped child process 1931, status = succeeded6948worker1 # [ 196.940845] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526949worker1 # [ 196.950150] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.562Z level=INFO msg="All 2 paths already cached"6950worker1 # [ 196.959057] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392526951worker1 # [ 196.975578] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6952worker1 # [ 197.109314] nix-daemon[833]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6953worker1 # [ 197.133309] nix-daemon[833]: reaped child process 1959, status = succeeded6954worker1 # [ 197.234576] nix-daemon[833]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)6955worker1 # [ 197.252110] nix-daemon[833]: reaped child process 1964, status = succeeded6956lb # [ 197.324428] niks3-server[1185]: 2026/09/22 13:48:54 INFO Received uploads request method=POST path=/api/pending_closures6957worker1 # [ 197.266818] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.878Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6958worker1 # [ 197.268456] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.879Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6959lb # [ 197.352271] niks3-server[1185]: 2026/09/22 13:48:54 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6960worker1 # [ 197.301859] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.914Z level=INFO msg="Uploading 1 narinfos"6961lb # [ 197.368820] niks3-server[1185]: 2026/09/22 13:48:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6962lb # [ 197.377951] niks3-server[1185]: 2026/09/22 13:48:54 INFO Signed narinfos id=74 count=16963lb # [ 197.385865] niks3-server[1185]: 2026/09/22 13:48:54 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6964worker1 # [ 197.331775] nix-grpc-daemon[1733]: time=2026-09-22T13:48:54.944Z level=INFO msg="Upload complete. (344ms)"6965lb # [ 197.398337] niks3-server[1185]: 2026/09/22 13:48:55 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6966lb # [ 197.409772] niks3-server[1185]: 2026/09/22 13:48:55 INFO Completed upload id=746967worker1 # [ 197.341094] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=7526968lb # [ 197.412932] niks3-server[1185]: 2026/09/22 13:48:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6969worker1 # [ 197.353905] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526970worker1 # [ 197.359170] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6971worker1 # [ 197.363459] nix-daemon[833]: reaped child process 1954, status = succeeded6972worker1 # [ 197.373321] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6973worker1 # [ 198.439031] nix-daemon[833]: reaped child process 1968, status = succeeded6974worker1 # [ 198.554510] nix-daemon[833]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)6975worker1 # [ 198.575035] nix-daemon[833]: reaped child process 1987, status = succeeded6976lb # [ 198.651027] niks3-server[1185]: 2026/09/22 13:48:56 INFO Received uploads request method=POST path=/api/pending_closures6977worker1 # [ 198.593683] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.204Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6978worker1 # [ 198.595464] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.204Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6979lb # [ 198.687501] niks3-server[1185]: 2026/09/22 13:48:56 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6980lb # [ 198.701973] niks3-server[1185]: 2026/09/22 13:48:56 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6981worker1 # [ 198.651746] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.263Z level=INFO msg="Uploading 1 narinfos"6982lb # [ 198.717645] niks3-server[1185]: 2026/09/22 13:48:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6983lb # [ 198.722258] niks3-server[1185]: 2026/09/22 13:48:56 INFO Signed narinfos id=75 count=16984lb # [ 198.728601] niks3-server[1185]: 2026/09/22 13:48:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6985lb # [ 198.737586] niks3-server[1185]: 2026/09/22 13:48:56 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6986lb # [ 198.746003] niks3-server[1185]: 2026/09/22 13:48:56 INFO Completed upload id=756987worker1 # [ 198.677442] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.289Z level=INFO msg="Upload complete. (292ms)"6988lb # [ 198.749078] niks3-server[1185]: 2026/09/22 13:48:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6989worker2 # [ 198.699378] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:56Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16990worker1 # [ 198.689268] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=16991worker1 # [ 198.710730] nix-daemon[833]: reaped child process 1966, status = succeeded6992worker1 # [ 198.742201] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392526993worker1 # [ 198.756881] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392526994worker1 # [ 198.770277] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.382Z level=INFO msg="All 2 paths already cached"6995worker1 # [ 198.780070] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392526996worker1 # [ 198.793229] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6997worker1 # [ 198.944876] nix-daemon[833]: accepted connection from pid 1991, user nix-grpc-daemon (trusted)6998worker1 # [ 198.976843] nix-daemon[833]: reaped child process 1994, status = succeeded6999worker1 # [ 199.067803] nix-daemon[833]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)7000worker1 # [ 199.083801] nix-daemon[833]: reaped child process 1999, status = succeeded7001lb # [ 199.156992] niks3-server[1185]: 2026/09/22 13:48:56 INFO Received uploads request method=POST path=/api/pending_closures7002worker1 # [ 199.101049] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.713Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7003worker1 # [ 199.102536] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.713Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"7004lb # [ 199.187275] niks3-server[1185]: 2026/09/22 13:48:56 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst7005worker1 # [ 199.134521] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.746Z level=INFO msg="Uploading 1 narinfos"7006lb # [ 199.201485] niks3-server[1185]: 2026/09/22 13:48:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7007lb # [ 199.212193] niks3-server[1185]: 2026/09/22 13:48:56 INFO Signed narinfos id=76 count=17008lb # [ 199.214995] niks3-server[1185]: 2026/09/22 13:48:56 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls7009worker1 # [ 199.161641] nix-grpc-daemon[1733]: time=2026-09-22T13:48:56.774Z level=INFO msg="Upload complete. (352ms)"7010lb # [ 199.229054] niks3-server[1185]: 2026/09/22 13:48:56 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7011lb # [ 199.237822] niks3-server[1185]: 2026/09/22 13:48:56 INFO Completed upload id=767012worker1 # [ 199.170320] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=7527013lb # [ 199.239842] niks3-server[1185]: 2026/09/22 13:48:56 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo7014worker1 # [ 199.183007] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392527015worker1 # [ 199.188665] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)7016worker1 # [ 199.193413] nix-daemon[833]: reaped child process 1989, status = succeeded7017worker1 # [ 199.205567] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)7018worker1 # [ 200.653299] nix-daemon[833]: reaped child process 2003, status = succeeded7019worker1 # [ 200.767006] nix-daemon[833]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)7020worker1 # [ 200.783251] nix-daemon[833]: reaped child process 2022, status = succeeded7021lb # [ 200.857526] niks3-server[1185]: 2026/09/22 13:48:58 INFO Received uploads request method=POST path=/api/pending_closures7022worker1 # [ 200.801634] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.412Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7023worker1 # [ 200.803205] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.412Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"7024lb # [ 200.895100] niks3-server[1185]: 2026/09/22 13:48:58 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst7025lb # [ 200.913038] niks3-server[1185]: 2026/09/22 13:48:58 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv7026worker1 # [ 200.856356] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.468Z level=INFO msg="Uploading 1 narinfos"7027lb # [ 200.922434] niks3-server[1185]: 2026/09/22 13:48:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7028lb # [ 200.932173] niks3-server[1185]: 2026/09/22 13:48:58 INFO Signed narinfos id=77 count=17029lb # [ 200.937597] niks3-server[1185]: 2026/09/22 13:48:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls7030lb # [ 200.946627] niks3-server[1185]: 2026/09/22 13:48:58 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7031lb # [ 200.954202] niks3-server[1185]: 2026/09/22 13:48:58 INFO Completed upload id=777032worker1 # [ 200.886179] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.498Z level=INFO msg="Upload complete. (270ms)"7033lb # [ 200.957468] niks3-server[1185]: 2026/09/22 13:48:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo7034worker2 # [ 200.909117] nix-grpc-daemon[1756]: ts=2026-09-22T13:48:58Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=17035worker1 # [ 200.901254] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=17036worker1 # [ 200.920169] nix-daemon[833]: reaped child process 2001, status = succeeded7037worker1 # [ 200.949971] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392527038worker1 # [ 200.965895] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392527039worker1 # [ 200.981884] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.594Z level=INFO msg="All 2 paths already cached"7040worker1 # [ 200.992105] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392527041worker1 # [ 201.005879] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)7042worker1 # [ 201.154732] nix-daemon[833]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)7043worker1 # [ 201.178747] nix-daemon[833]: reaped child process 2029, status = succeeded7044worker1 # [ 201.276284] nix-daemon[833]: accepted connection from pid 2031, user nix-grpc-daemon (trusted)7045worker1 # [ 201.293469] nix-daemon[833]: reaped child process 2034, status = succeeded7046lb # [ 201.365139] niks3-server[1185]: 2026/09/22 13:48:58 INFO Received uploads request method=POST path=/api/pending_closures7047worker1 # [ 201.308252] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.920Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7048worker1 # [ 201.310146] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.922Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"7049lb # [ 201.395699] niks3-server[1185]: 2026/09/22 13:48:59 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst7050lb # [ 201.410205] niks3-server[1185]: 2026/09/22 13:48:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7051worker1 # [ 201.350069] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.962Z level=INFO msg="Uploading 1 narinfos"7052lb # [ 201.424386] niks3-server[1185]: 2026/09/22 13:48:59 INFO Signed narinfos id=78 count=17053lb # [ 201.427623] niks3-server[1185]: 2026/09/22 13:48:59 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls7054worker1 # [ 201.376722] nix-grpc-daemon[1733]: time=2026-09-22T13:48:58.989Z level=INFO msg="Upload complete. (355ms)"7055lb # [ 201.443369] niks3-server[1185]: 2026/09/22 13:48:59 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7056worker1 # [ 201.384141] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=0 paths=1 nar_bytes_in=7527057lb # [ 201.454809] niks3-server[1185]: 2026/09/22 13:48:59 INFO Completed upload id=787058lb # [ 201.460634] niks3-server[1185]: 2026/09/22 13:48:59 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7059worker1 # [ 201.394565] nix-grpc-daemon[1673]: ts=2026-09-22T13:48:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392527060worker1 # [ 201.405449] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)7061worker1 # [ 201.410323] nix-daemon[833]: reaped child process 2024, status = succeeded7062worker1 # [ 201.422781] nix-daemon[833]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)7063worker1 # [ 202.296690] nix-daemon[833]: reaped child process 2038, status = succeeded7064worker1 # [ 202.370872] nix-daemon[833]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)7065worker1 # [ 202.390376] nix-daemon[833]: reaped child process 2057, status = succeeded7066lb # [ 202.462637] niks3-server[1185]: 2026/09/22 13:49:00 INFO Received uploads request method=POST path=/api/pending_closures7067worker1 # [ 202.407539] nix-grpc-daemon[1733]: time=2026-09-22T13:49:00.017Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7068worker1 # [ 202.409273] nix-grpc-daemon[1733]: time=2026-09-22T13:49:00.017Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7069lb # [ 202.494502] niks3-server[1185]: 2026/09/22 13:49:00 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7070lb # [ 202.513335] niks3-server[1185]: 2026/09/22 13:49:00 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7071worker1 # [ 202.459167] nix-grpc-daemon[1733]: time=2026-09-22T13:49:00.071Z level=INFO msg="Uploading 1 narinfos"7072lb # [ 202.526860] niks3-server[1185]: 2026/09/22 13:49:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7073lb # [ 202.537539] niks3-server[1185]: 2026/09/22 13:49:00 INFO Signed narinfos id=79 count=17074lb # [ 202.543774] niks3-server[1185]: 2026/09/22 13:49:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7075lb # [ 202.555949] niks3-server[1185]: 2026/09/22 13:49:00 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7076worker1 # [ 202.495757] nix-grpc-daemon[1733]: time=2026-09-22T13:49:00.108Z level=INFO msg="Upload complete. (263ms)"7077lb # [ 202.563618] niks3-server[1185]: 2026/09/22 13:49:00 INFO Completed upload id=797078lb # [ 202.568185] niks3-server[1185]: 2026/09/22 13:49:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7079worker1 # [ 202.509191] nix-grpc-daemon[1673]: ts=2026-09-22T13:49:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39252 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=17080worker1 # [ 202.525705] nix-daemon[833]: reaped child process 2036, status = succeeded7081client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 19.18 seconds)7082worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7083worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7084client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7085client # [ 202.791402] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7086client # Running as unit: gpu.service; invocation ID: 6d2d4eaaa7ea4eb3bc1c33944581f3747087client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.08 seconds)7088lb: must succeed: curl -sf localhost:9901/clusters7089lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7090lb: must succeed: curl -sf localhost:9901/clusters7091lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7092worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7093worker1 # [ 202.943450] nix-grpc-daemon[1673]: ts=2026-09-22T13:49:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:392527094client # [ 203.003079] nix[1517]: don't know how to build these paths:7095client # [ 203.004162] nix[1517]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7096worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7097worker1 # [ 202.956550] nix-grpc-daemon[1673]: ts=2026-09-22T13:49:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392527098client # [ 203.032648] nix[1517]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker7099worker2 # [ 202.992574] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:00Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07100lb: must succeed: curl -sf localhost:9901/clusters7101lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7102lb: must succeed: curl -sf localhost:9901/clusters7103lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7104worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7105worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7106client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7107lb # [ 210.102551] envoy[913]: conn peer=[2001:db8:1::1]:59220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27174 tx=12491 ms=189017108client # [ 233.038452] nix[1517]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}7109client # [ 233.048437] nix[1517]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7110client # [ 233.090677] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7111client # [ 233.097450] systemd[1]: gpu.service: Failed with result 'exit-code'.7112client # [ 233.101268] systemd[1]: gpu.service: Consumed 206ms CPU time over 30.299s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7113client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.07 seconds)7114client: must succeed: journalctl -u gpu -o cat7115client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7116lb: must succeed: curl -sf localhost:9901/clusters7117lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7118lb: must succeed: curl -sf localhost:9901/clusters7119lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7120worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7121worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7122(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.98 seconds)7123subtest: low disk drains a worker and builds go to the other7124worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7125worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.20 seconds)7126worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7127worker1 # [ 235.146754] nix-grpc-daemon[1673]: ts=2026-09-22T13:49:32Z level=info event=unhealthy reason=min_free7128worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)7129client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27130client # don't know how to build these paths:7131client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7132worker1 # [ 235.933213] nix-grpc-daemon[1673]: ts=2026-09-22T13:49:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:392527133worker2 # [ 235.980263] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:33Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07134worker2 # [ 235.981054] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:33Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17135worker2 # [ 236.025410] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537187136worker2 # [ 236.043027] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537187137client # copying 1 paths to 'lb:50051'...7138client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7139worker2 # [ 236.060641] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537187140worker2 # [ 236.075889] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7141worker2 # [ 236.239786] nix-daemon[862]: accepted connection from pid 2044, user nix-grpc-daemon (trusted)7142worker2 # [ 236.262507] nix-daemon[862]: reaped child process 2047, status = succeeded7143worker2 # [ 236.367617] nix-daemon[862]: accepted connection from pid 2049, user nix-grpc-daemon (trusted)7144worker2 # [ 236.386225] nix-daemon[862]: reaped child process 2052, status = succeeded7145lb # [ 236.450458] niks3-server[1185]: 2026/09/22 13:49:34 INFO Received uploads request method=POST path=/api/pending_closures7146worker2 # [ 236.407271] nix-grpc-daemon[1830]: time=2026-09-22T13:49:34.005Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7147worker2 # [ 236.408945] nix-grpc-daemon[1830]: time=2026-09-22T13:49:34.005Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7148lb # [ 236.487337] niks3-server[1185]: 2026/09/22 13:49:34 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7149lb # [ 236.502578] niks3-server[1185]: 2026/09/22 13:49:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7150worker2 # [ 236.452687] nix-grpc-daemon[1830]: time=2026-09-22T13:49:34.050Z level=INFO msg="Uploading 1 narinfos"7151lb # [ 236.513751] niks3-server[1185]: 2026/09/22 13:49:34 INFO Signed narinfos id=80 count=17152lb # [ 236.519459] niks3-server[1185]: 2026/09/22 13:49:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7153lb # [ 236.532244] niks3-server[1185]: 2026/09/22 13:49:34 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7154lb # [ 236.540663] niks3-server[1185]: 2026/09/22 13:49:34 INFO Completed upload id=807155worker2 # [ 236.484646] nix-grpc-daemon[1830]: time=2026-09-22T13:49:34.083Z level=INFO msg="Upload complete. (391ms)"7156lb # [ 236.545174] niks3-server[1185]: 2026/09/22 13:49:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7157worker2 # [ 236.497334] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=0 paths=1 nar_bytes_in=5127158worker2 # [ 236.505211] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537187159worker2 # [ 236.517590] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7160worker2 # [ 236.522726] nix-daemon[862]: reaped child process 2042, status = succeeded7161worker2 # [ 236.533356] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7162client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7163client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7164worker1 # [ 235.941597] nix-grpc-daemon[1673]: ts=2026-09-22T13:49:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392527165client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7166client # farm-a> LOG-farm-a-drain7167worker2 # [ 237.405469] nix-daemon[862]: reaped child process 2056, status = succeeded7168worker2 # [ 237.515230] nix-daemon[862]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)7169worker2 # [ 237.533894] nix-daemon[862]: reaped child process 2075, status = succeeded7170lb # [ 237.598425] niks3-server[1185]: 2026/09/22 13:49:35 INFO Received uploads request method=POST path=/api/pending_closures7171worker2 # [ 237.556437] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.152Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7172worker2 # [ 237.558252] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.152Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7173lb # [ 237.648115] niks3-server[1185]: 2026/09/22 13:49:35 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7174lb # [ 237.658453] niks3-server[1185]: 2026/09/22 13:49:35 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7175worker2 # [ 237.616618] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.214Z level=INFO msg="Uploading 1 narinfos"7176lb # [ 237.673486] niks3-server[1185]: 2026/09/22 13:49:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7177lb # [ 237.684314] niks3-server[1185]: 2026/09/22 13:49:35 INFO Signed narinfos id=81 count=17178lb # [ 237.687076] niks3-server[1185]: 2026/09/22 13:49:35 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7179lb # [ 237.701879] niks3-server[1185]: 2026/09/22 13:49:35 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7180lb # [ 237.709990] niks3-server[1185]: 2026/09/22 13:49:35 INFO Completed upload id=817181lb # [ 237.712740] niks3-server[1185]: 2026/09/22 13:49:35 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7182worker2 # [ 237.652850] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.251Z level=INFO msg="Upload complete. (308ms)"7183worker2 # [ 237.671779] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=17184worker2 # [ 237.688374] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:35Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07185worker2 # [ 237.695142] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:35Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17186worker2 # [ 237.699800] nix-daemon[862]: reaped child process 2054, status = succeeded7187worker2 # [ 237.716740] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537187188worker2 # [ 237.731304] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537187189worker2 # [ 237.745263] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.343Z level=INFO msg="All 1 paths already cached"7190client # copying 1 paths to 'lb:50051'...7191client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7192worker2 # [ 237.756112] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537187193worker2 # [ 237.769388] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7194worker2 # [ 237.927712] nix-daemon[862]: accepted connection from pid 2081, user nix-grpc-daemon (trusted)7195worker2 # [ 237.946136] nix-daemon[862]: reaped child process 2084, status = succeeded7196worker2 # [ 238.048323] nix-daemon[862]: accepted connection from pid 2086, user nix-grpc-daemon (trusted)7197worker2 # [ 238.064272] nix-daemon[862]: reaped child process 2089, status = succeeded7198lb # [ 238.126067] niks3-server[1185]: 2026/09/22 13:49:35 INFO Received uploads request method=POST path=/api/pending_closures7199worker2 # [ 238.079512] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.677Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7200worker2 # [ 238.081248] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.677Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7201lb # [ 238.153884] niks3-server[1185]: 2026/09/22 13:49:35 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7202worker2 # [ 238.114211] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.712Z level=INFO msg="Uploading 1 narinfos"7203lb # [ 238.170514] niks3-server[1185]: 2026/09/22 13:49:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7204lb # [ 238.178807] niks3-server[1185]: 2026/09/22 13:49:35 INFO Signed narinfos id=82 count=17205lb # [ 238.186709] niks3-server[1185]: 2026/09/22 13:49:35 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7206lb # [ 238.197866] niks3-server[1185]: 2026/09/22 13:49:35 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7207worker2 # [ 238.149094] nix-grpc-daemon[1830]: time=2026-09-22T13:49:35.747Z level=INFO msg="Upload complete. (364ms)"7208lb # [ 238.207108] niks3-server[1185]: 2026/09/22 13:49:35 INFO Completed upload id=827209lb # [ 238.211396] niks3-server[1185]: 2026/09/22 13:49:35 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7210worker2 # [ 238.161142] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=0 paths=1 nar_bytes_in=6327211worker2 # [ 238.171969] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537187212worker2 # [ 238.178816] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7213worker2 # [ 238.183895] nix-daemon[862]: reaped child process 2079, status = succeeded7214worker2 # [ 238.194839] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7215client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7216client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7217client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7218client # farm-b> LOG-farm-b-drain7219worker2 # [ 238.823179] nix-daemon[862]: reaped child process 2093, status = succeeded7220worker2 # [ 238.918295] nix-daemon[862]: accepted connection from pid 2109, user nix-grpc-daemon (trusted)7221worker2 # [ 238.936182] nix-daemon[862]: reaped child process 2112, status = succeeded7222lb # [ 238.999264] niks3-server[1185]: 2026/09/22 13:49:36 INFO Received uploads request method=POST path=/api/pending_closures7223worker2 # [ 238.955790] nix-grpc-daemon[1830]: time=2026-09-22T13:49:36.552Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7224worker2 # [ 238.959473] nix-grpc-daemon[1830]: time=2026-09-22T13:49:36.552Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7225lb # [ 239.050095] niks3-server[1185]: 2026/09/22 13:49:36 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7226worker2 # [ 239.014073] nix-grpc-daemon[1830]: time=2026-09-22T13:49:36.612Z level=INFO msg="Uploading 1 narinfos"7227lb # [ 239.060395] niks3-server[1185]: 2026/09/22 13:49:36 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7228lb # [ 239.076318] niks3-server[1185]: 2026/09/22 13:49:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7229lb # [ 239.084272] niks3-server[1185]: 2026/09/22 13:49:36 INFO Signed narinfos id=83 count=17230lb # [ 239.088116] niks3-server[1185]: 2026/09/22 13:49:36 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7231worker2 # [ 239.041395] nix-grpc-daemon[1830]: time=2026-09-22T13:49:36.639Z level=INFO msg="Upload complete. (235ms)"7232lb # [ 239.097705] niks3-server[1185]: 2026/09/22 13:49:36 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7233lb # [ 239.107073] niks3-server[1185]: 2026/09/22 13:49:36 INFO Completed upload id=837234lb # [ 239.111478] niks3-server[1185]: 2026/09/22 13:49:36 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7235worker2 # [ 239.056876] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=0 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=17236worker2 # [ 239.079115] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:36Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07237worker2 # [ 239.084708] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:36Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17238worker2 # [ 239.089855] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537187239worker2 # [ 239.095667] nix-daemon[862]: reaped child process 2091, status = succeeded7240worker2 # [ 239.099758] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537187241worker2 # [ 239.108531] nix-grpc-daemon[1830]: time=2026-09-22T13:49:36.706Z level=INFO msg="All 2 paths already cached"7242client # copying 1 paths to 'lb:50051'...7243client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7244worker2 # [ 239.118667] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537187245worker2 # [ 239.132418] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7246worker2 # [ 239.270305] nix-daemon[862]: accepted connection from pid 2118, user nix-grpc-daemon (trusted)7247worker2 # [ 239.289126] nix-daemon[862]: reaped child process 2121, status = succeeded7248worker2 # [ 239.398657] nix-daemon[862]: accepted connection from pid 2123, user nix-grpc-daemon (trusted)7249worker2 # [ 239.420444] nix-daemon[862]: reaped child process 2126, status = succeeded7250lb # [ 239.474474] niks3-server[1185]: 2026/09/22 13:49:37 INFO Received uploads request method=POST path=/api/pending_closures7251worker2 # [ 239.429732] nix-grpc-daemon[1830]: time=2026-09-22T13:49:37.027Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7252worker2 # [ 239.431236] nix-grpc-daemon[1830]: time=2026-09-22T13:49:37.028Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7253lb # [ 239.512600] niks3-server[1185]: 2026/09/22 13:49:37 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7254worker2 # [ 239.471542] nix-grpc-daemon[1830]: time=2026-09-22T13:49:37.069Z level=INFO msg="Uploading 1 narinfos"7255lb # [ 239.527023] niks3-server[1185]: 2026/09/22 13:49:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7256lb # [ 239.537814] niks3-server[1185]: 2026/09/22 13:49:37 INFO Signed narinfos id=84 count=17257lb # [ 239.544430] niks3-server[1185]: 2026/09/22 13:49:37 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7258lb # [ 239.556772] niks3-server[1185]: 2026/09/22 13:49:37 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7259lb # [ 239.567138] niks3-server[1185]: 2026/09/22 13:49:37 INFO Completed upload id=847260worker2 # [ 239.510539] nix-grpc-daemon[1830]: time=2026-09-22T13:49:37.109Z level=INFO msg="Upload complete. (368ms)"7261lb # [ 239.570921] niks3-server[1185]: 2026/09/22 13:49:37 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7262worker2 # [ 239.520278] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=0 paths=1 nar_bytes_in=7607263worker2 # [ 239.536574] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537187264worker2 # [ 239.543589] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7265worker2 # [ 239.549205] nix-daemon[862]: reaped child process 2116, status = succeeded7266client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7267worker2 # [ 239.562298] nix-daemon[862]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)7268client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7269lb # [ 240.103081] envoy[913]: conn peer=[2001:db8:1::1]:48060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300697270client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7271client # farm-top> LOG-farm-top-drain7272worker2 # [ 241.269167] nix-daemon[862]: reaped child process 2130, status = succeeded7273worker2 # [ 241.340465] nix-daemon[862]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)7274worker2 # [ 241.359009] nix-daemon[862]: reaped child process 2149, status = succeeded7275lb # [ 241.422413] niks3-server[1185]: 2026/09/22 13:49:39 INFO Received uploads request method=POST path=/api/pending_closures7276worker2 # [ 241.374361] nix-grpc-daemon[1830]: time=2026-09-22T13:49:38.971Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7277worker2 # [ 241.376078] nix-grpc-daemon[1830]: time=2026-09-22T13:49:38.971Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7278lb # [ 241.450013] niks3-server[1185]: 2026/09/22 13:49:39 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7279lb # [ 241.464167] niks3-server[1185]: 2026/09/22 13:49:39 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7280worker2 # [ 241.415334] nix-grpc-daemon[1830]: time=2026-09-22T13:49:39.013Z level=INFO msg="Uploading 1 narinfos"7281lb # [ 241.472263] niks3-server[1185]: 2026/09/22 13:49:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7282lb # [ 241.478137] niks3-server[1185]: 2026/09/22 13:49:39 INFO Signed narinfos id=85 count=17283lb # [ 241.482322] niks3-server[1185]: 2026/09/22 13:49:39 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7284lb # [ 241.488844] niks3-server[1185]: 2026/09/22 13:49:39 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7285lb # [ 241.494022] niks3-server[1185]: 2026/09/22 13:49:39 INFO Completed upload id=857286worker2 # [ 241.436780] nix-grpc-daemon[1830]: time=2026-09-22T13:49:39.035Z level=INFO msg="Upload complete. (263ms)"7287lb # [ 241.496088] niks3-server[1185]: 2026/09/22 13:49:39 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7288worker2 # [ 241.449062] nix-grpc-daemon[1756]: ts=2026-09-22T13:49:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53718 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=17289worker2 # [ 241.467090] nix-daemon[862]: reaped child process 2128, status = succeeded7290client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.77 seconds)7291client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7292client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7293worker1: must succeed: rm /nix/.rw-store/fill7294worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7295worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7296worker1 # [ 242.149024] nix-grpc-daemon[1673]: ts=2026-09-22T13:49:39Z level=info event=healthy reason=min_free7297worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7298(finished: subtest: low disk drains a worker and builds go to the other, in 9.58 seconds)7299(finished: run the VM test script, in 243.92 seconds)7300test script finished in 244.40s7301cleanup7302kill QemuMachine (pid 49)7303client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7304client # [2026-09-22T13:49:40Z INFO virtiofsd] Client disconnected, shutting down7305client # [2026-09-22T13:49:40Z INFO virtiofsd] Client disconnected, shutting down7306client # [2026-09-22T13:49:40Z INFO virtiofsd] Client disconnected, shutting down7307kill QemuMachine (pid 50)7308lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7309lb # [2026-09-22T13:49:40Z INFO virtiofsd] Client disconnected, shutting down7310lb # [2026-09-22T13:49:40Z INFO virtiofsd] Client disconnected, shutting down7311lb # [2026-09-22T13:49:40Z INFO virtiofsd] Client disconnected, shutting down7312kill QemuMachine (pid 51)7313worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7314worker1 # [2026-09-22T13:49:41Z INFO virtiofsd] Client disconnected, shutting down7315worker1 # [2026-09-22T13:49:41Z INFO virtiofsd] Client disconnected, shutting down7316worker1 # [2026-09-22T13:49:41Z INFO virtiofsd] Client disconnected, shutting down7317kill QemuMachine (pid 52)7318worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7319worker2 # [2026-09-22T13:49:41Z INFO virtiofsd] Client disconnected, shutting down7320worker2 # [2026-09-22T13:49:41Z INFO virtiofsd] Client disconnected, shutting down7321worker2 # [2026-09-22T13:49:41Z INFO virtiofsd] Client disconnected, shutting down7322(finished: cleanup, in 1.15 seconds)7323additionally exposed symbols:7324 client, lb, worker1, worker2,7325 vlan1,7326 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh