vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #493
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Formatting '/build/vm-state-worker1/tmp.SKpXJvGDa2', fmt=raw size=107374182414worker1: QEMU running (pid 51)15lb # Disk image does not exist, creating the virtualisation disk image...16lb: QEMU running (pid 50)17worker1 # mke2fs 1.47.4 (6-Mar-2025)18lb # Formatting '/build/vm-state-lb/tmp.exMiGMgJUS', fmt=raw size=107374182419client: QEMU running (pid 49)20client # Disk image does not exist, creating the virtualisation disk image...21worker2: QEMU running (pid 52)22worker2 # Disk image does not exist, creating the virtualisation disk image...23client # Formatting '/build/vm-state-client/tmp.bGFFMN1F6n', fmt=raw size=107374182424worker1 # Discarding device blocks: 0/262144 done25lb # mke2fs 1.47.4 (6-Mar-2025)26(finished: start all VMs, in 0.50 seconds)27worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes28lb # Discarding device blocks: 0/262144 done29client: waiting for unit network-addresses-eth1.service30worker1 # Filesystem UUID: b5ce059f-4a59-46b4-bd3f-48aa5e04944e31lb # Creating filesystem with 262144 4k blocks and 65536 inodes32client: waiting for the VM to finish booting33client # mke2fs 1.47.4 (6-Mar-2025)34worker1 # Superblock backups stored on blocks:35worker2 # Formatting '/build/vm-state-worker2/tmp.P3YjaNsM29', fmt=raw size=107374182436lb # Filesystem UUID: ad359d95-b0e5-48b9-a22f-93ffc0000a4d37worker2 # mke2fs 1.47.4 (6-Mar-2025)38lb # Superblock backups stored on blocks:39worker1 # 32768, 98304, 163840, 22937640client # Discarding device blocks: 0/262144 done41worker2 # Discarding device blocks: 0/262144 done42lb # 32768, 98304, 163840, 22937643worker1 # 44client # Creating filesystem with 262144 4k blocks and 65536 inodes45worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes46lb # 47worker1 # Allocating group tables: 0/8 done48client # Filesystem UUID: 449ab7fb-d156-4db5-9325-439d708b038349worker2 # Filesystem UUID: 386cf252-64d3-4618-a226-465d74eae13650lb # Allocating group tables: 0/8 done51worker1 # Writing inode tables: 0/8 done52client # Superblock backups stored on blocks:53worker2 # Superblock backups stored on blocks:54client # 32768, 98304, 163840, 22937655worker1 # Creating journal (8192 blocks): done56lb # Writing inode tables: 0/8 done57client # 58worker1 # Writing superblocks and filesystem accounting information: 0/8 done59worker2 # 32768, 98304, 163840, 22937660worker1 # 61worker2 # 62client # Allocating group tables: 0/8 done63lb # Creating journal (8192 blocks): done64worker1 # Virtualisation disk image created.65worker2 # Allocating group tables: 0/8 done66client # Writing inode tables: 0/8 done67lb # Writing superblocks and filesystem accounting information: 0/8 done68worker1 # Starting virtiofs daemons...69worker2 # Writing inode tables: 0/8 done70client # Creating journal (8192 blocks): done71lb # 72worker1 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)73worker2 # Creating journal (8192 blocks): done74client # Writing superblocks and filesystem accounting information: 0/8 done75lb # Virtualisation disk image created.76worker1 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77worker2 # Writing superblocks and filesystem accounting information: 0/8 done78client # 79lb # Starting virtiofs daemons...80worker1 # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...81worker2 # 82client # Virtualisation disk image created.83lb # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84worker1 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker2 # Virtualisation disk image created.86client # Starting virtiofs daemons...87lb # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether88worker1 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # Starting virtiofs daemons...90client # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91lb # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...92worker1 # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker2 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94client # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95lb # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)96worker1 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)97worker2 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether98client # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)99lb # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether100worker1 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101worker2 # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...102client # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether103lb # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...104worker1 # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...105worker2 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)106client # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)107lb # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)108worker1 # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests109worker2 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether110client # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether111lb # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether112worker1 # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests113worker2 # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...114client # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...115lb # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...116worker1 # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests117worker2 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)118client # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...119worker2 # [2026-09-22T10:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether120client # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...121worker2 # [2026-09-22T10:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...122client # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests123worker2 # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests124client # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests125worker2 # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests126client # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests127worker2 # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests128lb # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-22T10:58:24Z INFO virtiofsd] Client connected, servicing requests131worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker2 # [ 0.000000] KASLR enabled134worker2 # [ 0.000000] random: crng init done135worker2 # [ 0.000000] Machine model: linux,dummy-virt136worker2 # [ 0.000000] efi: UEFI not found.137worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140worker2 # [ 0.000000] Zone ranges:141worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142worker2 # [ 0.000000] DMA32 empty143worker2 # [ 0.000000] Normal empty144worker2 # [ 0.000000] Device empty145worker2 # [ 0.000000] Movable zone start for each node146worker2 # [ 0.000000] Early memory node ranges147worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker2 # [ 0.000000] psci: probing for conduit method from DT.151worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153worker2 # [ 0.000000] psci: Trusted OS migration not required154worker2 # [ 0.000000] psci: SMC Calling Convention v1.1155worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157worker2 # [ 0.000000] Detected PIPT I-cache on CPU0158worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface160worker2 # [ 0.000000] CPU features: detected: Spectre-v4161worker2 # [ 0.000000] CPU features: detected: Spectre-BHB162lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]163worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38164worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23165lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026166worker2 # [ 0.000000] alternatives: applying boot alternatives167lb # [ 0.000000] KASLR enabled168lb # [ 0.000000] random: crng init done169lb # [ 0.000000] Machine model: linux,dummy-virt170lb # [ 0.000000] efi: UEFI not found.171lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT172lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]173lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]174worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n25zl0ms12n9s2746s41c1gc7jq6ai9z-nixos-system-worker2-test/init regInfo=/nix/store/4aixqcvid266vaphk497m20cjsylkwqq-closure-info/registration console=ttyAMA0,115200n8 console=tty0175lb # [ 0.000000] Zone ranges:176lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]177lb # [ 0.000000] DMA32 empty178lb # [ 0.000000] Normal empty179worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4aixqcvid266vaphk497m20cjsylkwqq-closure-info/registration", will be passed to user space.180lb # [ 0.000000] Device empty181lb # [ 0.000000] Movable zone start for each node182worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes183lb # [ 0.000000] Early memory node ranges184worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)185lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]186worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)187lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]188lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000189worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB190worker2 # [ 0.000000] software IO TLB: area num 1.191lb # [ 0.000000] psci: probing for conduit method from DT.192lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.193worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB194lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs195worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)196lb # [ 0.000000] psci: Trusted OS migration not required197worker2 # [ 0.000000] Fallback order for Node 0: 0198lb # [ 0.000000] psci: SMC Calling Convention v1.1199worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216200worker2 # [ 0.000000] Policy zone: DMA201lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)202worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off203lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296204lb # [ 0.000000] Detected PIPT I-cache on CPU0205worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1206worker2 # [ 0.000000] allocated 3145728 bytes of page_ext207lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)208worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages209lb # [ 0.000000] CPU features: detected: GICv3 CPU interface210worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups211lb # [ 0.000000] CPU features: detected: Spectre-v4212worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.213lb # [ 0.000000] CPU features: detected: Spectre-BHB214worker2 # [ 0.000000] rcu: RCU event tracing is enabled.215lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38216worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.217lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23218worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.219worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]220lb # [ 0.000000] alternatives: applying boot alternatives221worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.222worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.223worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.224worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026225worker1 # [ 0.000000] KASLR enabled226worker1 # [ 0.000000] random: crng init done227worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1228worker1 # [ 0.000000] Machine model: linux,dummy-virt229worker1 # [ 0.000000] efi: UEFI not found.230worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.231client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]232lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hw6sjiya3fqcxqx5sjqhqri1ayky0v1l-nixos-system-lb-test/init regInfo=/nix/store/icv2rkyzy36dhd3gwpgrrp51zhv2f8w0-closure-info/registration console=ttyAMA0,115200n8 console=tty0233worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT234worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.235worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]236client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026237client # [ 0.000000] KASLR enabled238worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]239worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.240client # [ 0.000000] random: crng init done241worker1 # [ 0.000000] Zone ranges:242lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/icv2rkyzy36dhd3gwpgrrp51zhv2f8w0-closure-info/registration", will be passed to user space.243worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0244client # [ 0.000000] Machine model: linux,dummy-virt245worker2 # [ 0.000000] GICv3: 256 SPIs implemented246worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]247client # [ 0.000000] efi: UEFI not found.248lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes249worker1 # [ 0.000000] DMA32 empty250worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented251worker1 # [ 0.000000] Normal empty252client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT253worker1 # [ 0.000000] Device empty254worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq255lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)256worker1 # [ 0.000000] Movable zone start for each node257worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI258client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]259lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)260worker1 # [ 0.000000] Early memory node ranges261worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0262client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]263worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]264client # [ 0.000000] Zone ranges:265lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB266worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000267lb # [ 0.000000] software IO TLB: area num 1.268client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]269worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]270worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]271client # [ 0.000000] DMA32 empty272lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB273worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000274client # [ 0.000000] Normal empty275worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)276client # [ 0.000000] Device empty277worker1 # [ 0.000000] psci: probing for conduit method from DT.278lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)279client # [ 0.000000] Movable zone start for each node280worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.281lb # [ 0.000000] Fallback order for Node 0: 0282client # [ 0.000000] Early memory node ranges283worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)284worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs285client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]286lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216287worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000288worker1 # [ 0.000000] psci: Trusted OS migration not required289lb # [ 0.000000] Policy zone: DMA290worker1 # [ 0.000000] psci: SMC Calling Convention v1.1291client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]292worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000293lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off294client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000295worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.296worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)297lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1298client # [ 0.000000] psci: probing for conduit method from DT.299lb # [ 0.000000] allocated 3145728 bytes of page_ext300worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296301client # [ 0.000000] psci: PSCIv1.3 detected in firmware.302worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns303lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages304worker1 # [ 0.000000] Detected PIPT I-cache on CPU0305client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs306worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).307lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups308client # [ 0.000000] psci: Trusted OS migration not required309worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)310lb # [ 0.000000] rcu: Hierarchical RCU implementation.311client # [ 0.000000] psci: SMC Calling Convention v1.1312lb # [ 0.000000] rcu: RCU event tracing is enabled.313worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface314worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns315worker1 # [ 0.000000] CPU features: detected: Spectre-v4316client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)317lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.318worker1 # [ 0.000000] CPU features: detected: Spectre-BHB319worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns320lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.321client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296322worker2 # [ 0.000036] arm-pv: using stolen time PV323worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38324client # [ 0.000000] Detected PIPT I-cache on CPU0325lb # [ 0.000000] Rude variant of Tasks RCU enabled.326worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23327lb # [ 0.000000] Tracing variant of Tasks RCU enabled.328worker2 # [ 0.000687] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)329client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)330worker1 # [ 0.000000] alternatives: applying boot alternatives331worker2 # [ 0.000870] Console: colour dummy device 80x25332lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.333client # [ 0.000000] CPU features: detected: GICv3 CPU interface334worker2 # [ 0.000878] printk: legacy console [tty0] enabled335client # [ 0.000000] CPU features: detected: Spectre-v4336lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1337client # [ 0.000000] CPU features: detected: Spectre-BHB338worker2 # [ 0.001100] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)339lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.340client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38341worker2 # [ 0.001107] pid_max: default: 32768 minimum: 301342client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23343worker2 # [ 0.001189] LSM: initializing lsm=capability,landlock,yama,bpf,ima344lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.345client # [ 0.000000] alternatives: applying boot alternatives346worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c7rqk0rlr4l2j2rnw1rm976l59blicb7-nixos-system-worker1-test/init regInfo=/nix/store/56r3zx0w60z363pcikqhfcwz8a6gnw3f-closure-info/registration console=ttyAMA0,115200n8 console=tty0347worker2 # [ 0.001349] landlock: Up and running.348worker2 # [ 0.001352] Yama: becoming mindful.349lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.350worker2 # [ 0.002019] LSM support for eBPF active351lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0352worker2 # [ 0.002189] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)353lb # [ 0.000000] GICv3: 256 SPIs implemented354worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/56r3zx0w60z363pcikqhfcwz8a6gnw3f-closure-info/registration", will be passed to user space.355lb # [ 0.000000] GICv3: 0 Extended SPIs implemented356worker2 # [ 0.002229] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)357lb # [ 0.000000] Root IRQ handler: gic_handle_irq358worker2 # [ 0.004059] rcu: Hierarchical SRCU implementation.359worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes360lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI361client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test/init regInfo=/nix/store/s0il4g14jhd7gjas9f6wd09h5hkggkfy-closure-info/registration console=ttyAMA0,115200n8 console=tty0362worker2 # [ 0.004064] rcu: Max phase no-delay instances is 1000.363lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0364worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)365worker2 # [ 0.005357] fsl-mc MSI: its@8080000 domain created366lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000367worker2 # [ 0.005448] EFI services will not be available.368worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)369lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]370worker2 # [ 0.005524] smp: Bringing up secondary CPUs ...371worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB372worker2 # [ 0.005532] smp: Brought up 1 node, 1 CPU373client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/s0il4g14jhd7gjas9f6wd09h5hkggkfy-closure-info/registration", will be passed to user space.374worker1 # [ 0.000000] software IO TLB: area num 1.375lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)376worker2 # [ 0.005535] SMP: Total of 1 processors activated.377client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes378worker2 # [ 0.005540] CPU: All CPU(s) started at EL1379worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB380lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)381worker2 # [ 0.005550] CPU features: detected: Branch Target Identification382client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)383worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)384lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000385worker2 # [ 0.005556] CPU features: detected: ARMv8.4 Translation Table Level386worker1 # [ 0.000000] Fallback order for Node 0: 0387client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)388lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000389worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216390worker2 # [ 0.005559] CPU features: detected: Instruction cache invalidation not required for I/D coherence391client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB392worker1 # [ 0.000000] Policy zone: DMA393lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.394client # [ 0.000000] software IO TLB: area num 1.395worker2 # [ 0.005562] CPU features: detected: Data cache clean to the PoU not required for I/D coherence396worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off397client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB398lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns399worker2 # [ 0.005566] CPU features: detected: Common not Private translations400worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1401client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)402worker2 # [ 0.005570] CPU features: detected: CRC32 instructions403lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).404worker1 # [ 0.000000] allocated 3145728 bytes of page_ext405client # [ 0.000000] Fallback order for Node 0: 0406worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages407worker2 # [ 0.005572] CPU features: detected: Data cache clean to Point of Deep Persistence408client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216409worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups410lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns411client # [ 0.000000] Policy zone: DMA412worker2 # [ 0.005576] CPU features: detected: Data cache clean to Point of Persistence413worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.414lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns415client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off416worker1 # [ 0.000000] rcu: RCU event tracing is enabled.417worker2 # [ 0.005579] CPU features: detected: Data independent timing control (DIT)418lb # [ 0.000033] arm-pv: using stolen time PV419worker2 # [ 0.005582] CPU features: detected: E0PD420worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.421client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1422worker2 # [ 0.005585] CPU features: detected: Enhanced Counter Virtualization423client # [ 0.000000] allocated 3145728 bytes of page_ext424worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.425lb # [ 0.000545] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)426worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.427client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages428lb # [ 0.000741] Console: colour dummy device 80x25429worker2 # [ 0.005588] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)430worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.431client # [ 0.000000] ftrace: allocated 294 pages with 4 groups432lb # [ 0.000749] printk: legacy console [tty0] enabled433worker2 # [ 0.005592] CPU features: detected: Enhanced Virtualization Traps434client # [ 0.000000] rcu: Hierarchical RCU implementation.435worker2 # [ 0.005595] CPU features: detected: Fine Grained Traps436worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.437client # [ 0.000000] rcu: RCU event tracing is enabled.438lb # [ 0.000962] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)439worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1440worker2 # [ 0.005599] CPU features: detected: Generic authentication (architected QARMA5 algorithm)441lb # [ 0.000969] pid_max: default: 32768 minimum: 301442client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.443client # [ 0.000000] Trampoline variant of Tasks RCU enabled.444lb # [ 0.001048] LSM: initializing lsm=capability,landlock,yama,bpf,ima445worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.446worker2 # [ 0.005604] CPU features: detected: RCpc load-acquire (LDAPR)447lb # [ 0.001198] landlock: Up and running.448client # [ 0.000000] Rude variant of Tasks RCU enabled.449worker2 # [ 0.005607] CPU features: detected: LSE atomic instructions450lb # [ 0.001202] Yama: becoming mindful.451client # [ 0.000000] Tracing variant of Tasks RCU enabled.452worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.453worker2 # [ 0.005610] CPU features: detected: Privileged Access Never454lb # [ 0.001741] LSM support for eBPF active455worker2 # [ 0.005612] CPU features: detected: PMUv3456client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.457worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.458lb # [ 0.001909] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)459worker2 # [ 0.005615] CPU features: detected: RAS Extension Support460client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1461worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0462worker2 # [ 0.005618] CPU features: detected: RASv1p1 Extension Support463lb # [ 0.001942] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)464worker1 # [ 0.000000] GICv3: 256 SPIs implemented465worker2 # [ 0.005621] CPU features: detected: Random Number Generator466lb # [ 0.003786] rcu: Hierarchical SRCU implementation.467client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.468worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented469lb # [ 0.003790] rcu: Max phase no-delay instances is 1000.470worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq471client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.472lb # [ 0.005089] fsl-mc MSI: its@8080000 domain created473worker2 # [ 0.005623] CPU features: detected: Speculation barrier (SB)474worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI475lb # [ 0.005187] EFI services will not be available.476worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0477worker2 # [ 0.005626] CPU features: detected: Stage-2 Force Write-Back478lb # [ 0.005255] smp: Bringing up secondary CPUs ...479client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.480worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000481lb # [ 0.005264] smp: Brought up 1 node, 1 CPU482worker2 # [ 0.005629] CPU features: detected: TLB range maintenance instructions483client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0484worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]485lb # [ 0.005267] SMP: Total of 1 processors activated.486client # [ 0.000000] GICv3: 256 SPIs implemented487worker2 # [ 0.005633] CPU features: detected: Speculative Store Bypassing Safe (SSBS)488lb # [ 0.005272] CPU: All CPU(s) started at EL1489client # [ 0.000000] GICv3: 0 Extended SPIs implemented490worker2 # [ 0.005675] alternatives: applying system-wide alternatives491worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)492client # [ 0.000000] Root IRQ handler: gic_handle_irq493lb # [ 0.005283] CPU features: detected: Branch Target Identification494worker2 # [ 0.008728] CPU features: detected: BBM Level 2 without TLB conflict abort495client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI496lb # [ 0.005288] CPU features: detected: ARMv8.4 Translation Table Level497worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)498client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0499worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000500lb # [ 0.005291] CPU features: detected: Instruction cache invalidation not required for I/D coherence501client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000502worker2 # [ 0.008926] Memory: 1405880K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)503client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]504worker2 # [ 0.009311] devtmpfs: initialized505worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000506lb # [ 0.005295] CPU features: detected: Data cache clean to the PoU not required for I/D coherence507worker2 # [ 0.011127] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)508lb # [ 0.005298] CPU features: detected: Common not Private translations509client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)510worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.511lb # [ 0.005302] CPU features: detected: CRC32 instructions512worker2 # [ 0.011149] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).513worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns514client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)515lb # [ 0.005304] CPU features: detected: Data cache clean to Point of Deep Persistence516worker2 # [ 0.011325] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL517client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000518worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).519worker2 # [ 0.011329] 0 pages in range for non-PLT usage520lb # [ 0.005308] CPU features: detected: Data cache clean to Point of Persistence521worker2 # [ 0.011331] 508272 pages in range for PLT usage522client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000523worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns524lb # [ 0.005311] CPU features: detected: Data independent timing control (DIT)525worker2 # [ 0.011447] pinctrl core: initialized pinctrl subsystem526client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.527lb # [ 0.005314] CPU features: detected: E0PD528worker2 # [ 0.012236] DMI not present or invalid.529worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns530lb # [ 0.005317] CPU features: detected: Enhanced Counter Virtualization531worker2 # [ 0.015839] NET: Registered PF_NETLINK/PF_ROUTE protocol family532worker1 # [ 0.000029] arm-pv: using stolen time PV533client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns534worker2 # [ 0.018140] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations535lb # [ 0.005320] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)536client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).537worker1 # [ 0.000425] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)538lb # [ 0.005323] CPU features: detected: Enhanced Virtualization Traps539worker2 # [ 0.018320] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations540worker1 # [ 0.000599] Console: colour dummy device 80x25541lb # [ 0.005327] CPU features: detected: Fine Grained Traps542worker1 # [ 0.000606] printk: legacy console [tty0] enabled543client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns544worker2 # [ 0.018533] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations545lb # [ 0.005331] CPU features: detected: Generic authentication (architected QARMA5 algorithm)546worker2 # [ 0.018555] audit: initializing netlink subsys (disabled)547client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns548worker1 # [ 0.000794] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)549client # [ 0.000030] arm-pv: using stolen time PV550lb # [ 0.005335] CPU features: detected: RCpc load-acquire (LDAPR)551worker2 # [ 0.019122] thermal_sys: Registered thermal governor 'fair_share'552worker1 # [ 0.000800] pid_max: default: 32768 minimum: 301553lb # [ 0.005338] CPU features: detected: LSE atomic instructions554worker2 # [ 0.019124] thermal_sys: Registered thermal governor 'bang_bang'555worker1 # [ 0.000872] LSM: initializing lsm=capability,landlock,yama,bpf,ima556client # [ 0.000384] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)557lb # [ 0.005341] CPU features: detected: Privileged Access Never558worker1 # [ 0.001018] landlock: Up and running.559worker2 # [ 0.019127] thermal_sys: Registered thermal governor 'step_wise'560client # [ 0.000552] Console: colour dummy device 80x25561lb # [ 0.005344] CPU features: detected: PMUv3562worker1 # [ 0.001021] Yama: becoming mindful.563client # [ 0.000559] printk: legacy console [tty0] enabled564lb # [ 0.005347] CPU features: detected: RAS Extension Support565worker1 # [ 0.001491] LSM support for eBPF active566worker2 # [ 0.019130] thermal_sys: Registered thermal governor 'user_space'567worker2 # [ 0.019136] thermal_sys: Registered thermal governor 'power_allocator'568lb # [ 0.005350] CPU features: detected: RASv1p1 Extension Support569worker1 # [ 0.001639] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)570client # [ 0.000757] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)571lb # [ 0.005353] CPU features: detected: Random Number Generator572worker2 # [ 0.019160] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1573client # [ 0.000763] pid_max: default: 32768 minimum: 301574worker1 # [ 0.001677] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)575worker2 # [ 0.019169] cpuidle: using governor ladder576lb # [ 0.005355] CPU features: detected: Speculation barrier (SB)577worker1 # [ 0.003526] rcu: Hierarchical SRCU implementation.578worker2 # [ 0.019175] cpuidle: using governor menu579client # [ 0.000871] LSM: initializing lsm=capability,landlock,yama,bpf,ima580client # [ 0.001010] landlock: Up and running.581lb # [ 0.005358] CPU features: detected: Stage-2 Force Write-Back582worker1 # [ 0.003530] rcu: Max phase no-delay instances is 1000.583client # [ 0.001013] Yama: becoming mindful.584worker2 # [ 0.019367] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.585worker1 # [ 0.004795] fsl-mc MSI: its@8080000 domain created586lb # [ 0.005361] CPU features: detected: TLB range maintenance instructions587client # [ 0.001448] LSM support for eBPF active588worker2 # [ 0.019384] ASID allocator initialised with 65536 entries589worker1 # [ 0.004885] EFI services will not be available.590lb # [ 0.005366] CPU features: detected: Speculative Store Bypassing Safe (SSBS)591worker2 # [ 0.020691] Serial: AMBA PL011 UART driver592client # [ 0.001579] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)593worker1 # [ 0.004953] smp: Bringing up secondary CPUs ...594lb # [ 0.005404] alternatives: applying system-wide alternatives595worker1 # [ 0.004960] smp: Brought up 1 node, 1 CPU596worker2 # [ 0.026031] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1597client # [ 0.001612] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)598worker1 # [ 0.004964] SMP: Total of 1 processors activated.599lb # [ 0.008392] CPU features: detected: BBM Level 2 without TLB conflict abort600worker2 # [ 0.026163] printk: console [ttyAMA0] enabled601client # [ 0.003569] rcu: Hierarchical SRCU implementation.602worker1 # [ 0.004969] CPU: All CPU(s) started at EL1603client # [ 0.003575] rcu: Max phase no-delay instances is 1000.604worker1 # [ 0.004979] CPU features: detected: Branch Target Identification605client # [ 0.004852] fsl-mc MSI: its@8080000 domain created606lb # [ 0.008589] Memory: 1405664K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)607worker1 # [ 0.004984] CPU features: detected: ARMv8.4 Translation Table Level608client # [ 0.004942] EFI services will not be available.609lb # [ 0.008988] devtmpfs: initialized610client # [ 0.005008] smp: Bringing up secondary CPUs ...611worker1 # [ 0.004987] CPU features: detected: Instruction cache invalidation not required for I/D coherence612lb # [ 0.010809] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)613client # [ 0.005016] smp: Brought up 1 node, 1 CPU614client # [ 0.005019] SMP: Total of 1 processors activated.615worker1 # [ 0.004990] CPU features: detected: Data cache clean to the PoU not required for I/D coherence616lb # [ 0.010832] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).617client # [ 0.005025] CPU: All CPU(s) started at EL1618worker1 # [ 0.004994] CPU features: detected: Common not Private translations619lb # [ 0.011028] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL620client # [ 0.005035] CPU features: detected: Branch Target Identification621worker1 # [ 0.004997] CPU features: detected: CRC32 instructions622client # [ 0.005040] CPU features: detected: ARMv8.4 Translation Table Level623worker1 # [ 0.005000] CPU features: detected: Data cache clean to Point of Deep Persistence624client # [ 0.005043] CPU features: detected: Instruction cache invalidation not required for I/D coherence625worker1 # [ 0.005004] CPU features: detected: Data cache clean to Point of Persistence626worker2 # [ 0.144535] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages627client # [ 0.005047] CPU features: detected: Data cache clean to the PoU not required for I/D coherence628worker1 # [ 0.005007] CPU features: detected: Data independent timing control (DIT)629worker2 # [ 0.144552] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page630worker1 # [ 0.005010] CPU features: detected: E0PD631client # [ 0.005051] CPU features: detected: Common not Private translations632client # [ 0.005054] CPU features: detected: CRC32 instructions633worker2 # [ 0.144558] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages634worker1 # [ 0.005012] CPU features: detected: Enhanced Counter Virtualization635worker2 # [ 0.144562] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page636client # [ 0.005057] CPU features: detected: Data cache clean to Point of Deep Persistence637worker1 # [ 0.005016] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)638worker2 # [ 0.144566] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages639worker1 # [ 0.005019] CPU features: detected: Enhanced Virtualization Traps640client # [ 0.005060] CPU features: detected: Data cache clean to Point of Persistence641worker1 # [ 0.005022] CPU features: detected: Fine Grained Traps642worker2 # [ 0.144571] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page643client # [ 0.005063] CPU features: detected: Data independent timing control (DIT)644worker2 # [ 0.144575] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages645client # [ 0.005067] CPU features: detected: E0PD646worker1 # [ 0.005025] CPU features: detected: Generic authentication (architected QARMA5 algorithm)647worker2 # [ 0.144579] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page648client # [ 0.005069] CPU features: detected: Enhanced Counter Virtualization649worker1 # [ 0.005031] CPU features: detected: RCpc load-acquire (LDAPR)650lb # [ 0.011032] 0 pages in range for non-PLT usage651worker1 # [ 0.005034] CPU features: detected: LSE atomic instructions652client # [ 0.005072] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)653lb # [ 0.011033] 508272 pages in range for PLT usage654worker1 # [ 0.005037] CPU features: detected: Privileged Access Never655lb # [ 0.011142] pinctrl core: initialized pinctrl subsystem656client # [ 0.005076] CPU features: detected: Enhanced Virtualization Traps657worker1 # [ 0.005039] CPU features: detected: PMUv3658lb # [ 0.011946] DMI not present or invalid.659client # [ 0.005079] CPU features: detected: Fine Grained Traps660worker2 # [ 0.151858] fbcon: Taking over console661worker1 # [ 0.005042] CPU features: detected: RAS Extension Support662worker2 # [ 0.151873] ACPI: Interpreter disabled.663lb # [ 0.015265] NET: Registered PF_NETLINK/PF_ROUTE protocol family664worker1 # [ 0.005045] CPU features: detected: RASv1p1 Extension Support665client # [ 0.005082] CPU features: detected: Generic authentication (architected QARMA5 algorithm)666lb # [ 0.017461] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations667client # [ 0.005087] CPU features: detected: RCpc load-acquire (LDAPR)668worker1 # [ 0.005048] CPU features: detected: Random Number Generator669lb # [ 0.017640] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations670client # [ 0.005090] CPU features: detected: LSE atomic instructions671worker1 # [ 0.005050] CPU features: detected: Speculation barrier (SB)672client # [ 0.005093] CPU features: detected: Privileged Access Never673lb # [ 0.017838] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations674worker1 # [ 0.005053] CPU features: detected: Stage-2 Force Write-Back675client # [ 0.005096] CPU features: detected: PMUv3676lb # [ 0.017859] audit: initializing netlink subsys (disabled)677worker1 # [ 0.005056] CPU features: detected: TLB range maintenance instructions678client # [ 0.005099] CPU features: detected: RAS Extension Support679lb # [ 0.018370] thermal_sys: Registered thermal governor 'fair_share'680worker1 # [ 0.005061] CPU features: detected: Speculative Store Bypassing Safe (SSBS)681client # [ 0.005101] CPU features: detected: RASv1p1 Extension Support682lb # [ 0.018372] thermal_sys: Registered thermal governor 'bang_bang'683worker1 # [ 0.005099] alternatives: applying system-wide alternatives684client # [ 0.005104] CPU features: detected: Random Number Generator685lb # [ 0.018375] thermal_sys: Registered thermal governor 'step_wise'686worker1 # [ 0.008065] CPU features: detected: BBM Level 2 without TLB conflict abort687client # [ 0.005107] CPU features: detected: Speculation barrier (SB)688lb # [ 0.018378] thermal_sys: Registered thermal governor 'user_space'689client # [ 0.005109] CPU features: detected: Stage-2 Force Write-Back690lb # [ 0.018383] thermal_sys: Registered thermal governor 'power_allocator'691client # [ 0.005112] CPU features: detected: TLB range maintenance instructions692worker1 # [ 0.008218] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)693lb # [ 0.018407] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1694worker1 # [ 0.008588] devtmpfs: initialized695client # [ 0.005117] CPU features: detected: Speculative Store Bypassing Safe (SSBS)696lb # [ 0.018415] cpuidle: using governor ladder697worker2 # [ 0.160892] iommu: Default domain type: Translated698worker1 # [ 0.010348] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)699client # [ 0.005155] alternatives: applying system-wide alternatives700lb # [ 0.018421] cpuidle: using governor menu701worker2 # [ 0.160902] iommu: DMA domain TLB invalidation policy: strict mode702lb # [ 0.018599] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.703client # [ 0.008143] CPU features: detected: BBM Level 2 without TLB conflict abort704worker2 # [ 0.161319] SCSI subsystem initialized705worker1 # [ 0.010370] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).706lb # [ 0.018614] ASID allocator initialised with 65536 entries707lb # [ 0.019829] Serial: AMBA PL011 UART driver708worker2 # [ 0.163257] usbcore: registered new interface driver usbfs709client # [ 0.008307] Memory: 1405688K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)710client # [ 0.008680] devtmpfs: initialized711worker2 # [ 0.163289] usbcore: registered new interface driver hub712lb # [ 0.025116] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1713lb # [ 0.025241] printk: console [ttyAMA0] enabled714worker2 # [ 0.163305] usbcore: registered new device driver usb715client # [ 0.010464] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)716worker2 # [ 0.163572] pps_core: LinuxPPS API ver. 1 registered717lb # [ 0.146042] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages718worker2 # [ 0.163578] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>719lb # [ 0.146061] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page720worker2 # [ 0.163588] PTP clock support registered721worker2 # [ 0.163636] EDAC MC: Ver: 3.0.0722lb # [ 0.146067] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages723worker2 # [ 0.168208] scmi_core: SCMI protocol bus registered724lb # [ 0.146071] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page725worker2 # [ 0.169163] FPGA manager framework726lb # [ 0.146076] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages727worker2 # [ 0.170149] vgaarb: loaded728lb # [ 0.146080] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page729worker2 # [ 0.170779] clocksource: Switched to clocksource arch_sys_counter730lb # [ 0.146085] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages731worker1 # [ 0.010547] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL732worker1 # [ 0.010551] 0 pages in range for non-PLT usage733lb # [ 0.146089] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page734worker1 # [ 0.010552] 508272 pages in range for PLT usage735worker1 # [ 0.010665] pinctrl core: initialized pinctrl subsystem736worker1 # [ 0.011391] DMI not present or invalid.737worker1 # [ 0.014617] NET: Registered PF_NETLINK/PF_ROUTE protocol family738lb # [ 0.153607] fbcon: Taking over console739client # [ 0.010487] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).740worker2 # [ 0.174446] VFS: Disk quotas dquot_6.6.0741lb # [ 0.153621] ACPI: Interpreter disabled.742worker1 # [ 0.016790] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations743client # [ 0.010662] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL744worker2 # [ 0.174482] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)745lb # [ 0.155468] iommu: Default domain type: Translated746client # [ 0.010666] 0 pages in range for non-PLT usage747worker1 # [ 0.016976] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations748lb # [ 0.155478] iommu: DMA domain TLB invalidation policy: strict mode749client # [ 0.010667] 508272 pages in range for PLT usage750client # [ 0.010814] pinctrl core: initialized pinctrl subsystem751worker1 # [ 0.017176] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations752lb # [ 0.157123] SCSI subsystem initialized753client # [ 0.011611] DMI not present or invalid.754worker1 # [ 0.017199] audit: initializing netlink subsys (disabled)755client # [ 0.014797] NET: Registered PF_NETLINK/PF_ROUTE protocol family756worker1 # [ 0.017796] thermal_sys: Registered thermal governor 'fair_share'757worker2 # [ 0.178275] netfs: FS-Cache loaded758worker2 # [ 0.178401] pnp: PnP ACPI: disabled759client # [ 0.017001] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations760worker1 # [ 0.017798] thermal_sys: Registered thermal governor 'bang_bang'761worker1 # [ 0.017801] thermal_sys: Registered thermal governor 'step_wise'762client # [ 0.017183] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations763worker1 # [ 0.017804] thermal_sys: Registered thermal governor 'user_space'764client # [ 0.017380] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations765worker1 # [ 0.017810] thermal_sys: Registered thermal governor 'power_allocator'766client # [ 0.017400] audit: initializing netlink subsys (disabled)767worker1 # [ 0.017843] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1768client # [ 0.017951] thermal_sys: Registered thermal governor 'fair_share'769worker1 # [ 0.017852] cpuidle: using governor ladder770worker2 # [ 0.182373] NET: Registered PF_INET protocol family771client # [ 0.017953] thermal_sys: Registered thermal governor 'bang_bang'772worker1 # [ 0.017858] cpuidle: using governor menu773client # [ 0.017957] thermal_sys: Registered thermal governor 'step_wise'774worker2 # [ 0.182668] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)775worker1 # [ 0.018042] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.776client # [ 0.017959] thermal_sys: Registered thermal governor 'user_space'777worker1 # [ 0.018058] ASID allocator initialised with 65536 entries778worker1 # [ 0.019286] Serial: AMBA PL011 UART driver779client # [ 0.017965] thermal_sys: Registered thermal governor 'power_allocator'780lb # [ 0.164833] usbcore: registered new interface driver usbfs781lb # [ 0.164868] usbcore: registered new interface driver hub782client # [ 0.017989] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1783worker1 # [ 0.024539] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1784lb # [ 0.164892] usbcore: registered new device driver usb785client # [ 0.017998] cpuidle: using governor ladder786worker1 # [ 0.024686] printk: console [ttyAMA0] enabled787client # [ 0.018003] cpuidle: using governor menu788lb # [ 0.165183] pps_core: LinuxPPS API ver. 1 registered789client # [ 0.018182] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.790worker1 # [ 0.145691] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages791lb # [ 0.165190] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>792client # [ 0.018198] ASID allocator initialised with 65536 entries793lb # [ 0.165200] PTP clock support registered794worker1 # [ 0.145711] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page795client # [ 0.019414] Serial: AMBA PL011 UART driver796lb # [ 0.165238] EDAC MC: Ver: 3.0.0797worker1 # [ 0.145716] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages798lb # [ 0.169785] scmi_core: SCMI protocol bus registered799client # [ 0.024646] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1800worker1 # [ 0.145721] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page801client # [ 0.024794] printk: console [ttyAMA0] enabled802lb # [ 0.170709] FPGA manager framework803worker1 # [ 0.145725] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages804lb # [ 0.171730] vgaarb: loaded805worker1 # [ 0.145729] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page806client # [ 0.146086] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages807client # [ 0.146103] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page808worker1 # [ 0.145733] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages809lb # [ 0.172372] clocksource: Switched to clocksource arch_sys_counter810worker1 # [ 0.145737] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page811client # [ 0.146107] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages812lb # [ 0.172974] VFS: Disk quotas dquot_6.6.0813client # [ 0.146112] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page814lb # [ 0.172999] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)815client # [ 0.146116] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages816lb # [ 0.175339] netfs: FS-Cache loaded817lb # [ 0.175451] pnp: PnP ACPI: disabled818client # [ 0.146120] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page819worker1 # [ 0.153175] fbcon: Taking over console820worker1 # [ 0.153189] ACPI: Interpreter disabled.821client # [ 0.146124] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages822worker1 # [ 0.155056] iommu: Default domain type: Translated823client # [ 0.146128] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page824worker1 # [ 0.155066] iommu: DMA domain TLB invalidation policy: strict mode825worker1 # [ 0.156821] SCSI subsystem initialized826client # [ 0.153599] fbcon: Taking over console827client # [ 0.153614] ACPI: Interpreter disabled.828client # [ 0.155538] iommu: Default domain type: Translated829client # [ 0.155548] iommu: DMA domain TLB invalidation policy: strict mode830client # [ 0.157242] SCSI subsystem initialized831lb # [ 0.181514] NET: Registered PF_INET protocol family832lb # [ 0.181795] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)833worker1 # [ 0.163921] usbcore: registered new interface driver usbfs834worker1 # [ 0.163961] usbcore: registered new interface driver hub835worker1 # [ 0.163978] usbcore: registered new device driver usb836worker1 # [ 0.164251] pps_core: LinuxPPS API ver. 1 registered837client # [ 0.162466] usbcore: registered new interface driver usbfs838client # [ 0.162499] usbcore: registered new interface driver hub839worker1 # [ 0.164258] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>840client # [ 0.162515] usbcore: registered new device driver usb841worker1 # [ 0.164269] PTP clock support registered842worker1 # [ 0.164324] EDAC MC: Ver: 3.0.0843client # [ 0.162817] pps_core: LinuxPPS API ver. 1 registered844worker1 # [ 0.169058] scmi_core: SCMI protocol bus registered845client # [ 0.162823] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>846client # [ 0.162833] PTP clock support registered847worker1 # [ 0.170043] FPGA manager framework848client # [ 0.162883] EDAC MC: Ver: 3.0.0849worker1 # [ 0.171011] vgaarb: loaded850client # [ 0.167690] scmi_core: SCMI protocol bus registered851worker1 # [ 0.171652] clocksource: Switched to clocksource arch_sys_counter852client # [ 0.168664] FPGA manager framework853client # [ 0.169606] vgaarb: loaded854worker1 # [ 0.173699] VFS: Disk quotas dquot_6.6.0855client # [ 0.170245] clocksource: Switched to clocksource arch_sys_counter856worker1 # [ 0.173727] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)857worker2 # [ 0.213286] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)858worker1 # [ 0.177412] netfs: FS-Cache loaded859client # [ 0.174107] VFS: Disk quotas dquot_6.6.0860worker1 # [ 0.177527] pnp: PnP ACPI: disabled861worker2 # [ 0.213335] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)862client # [ 0.174135] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)863worker2 # [ 0.213368] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)864worker2 # [ 0.213448] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)865worker2 # [ 0.213593] TCP: Hash tables configured (established 16384 bind 16384)866client # [ 0.177810] netfs: FS-Cache loaded867worker2 # [ 0.213680] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)868client # [ 0.177926] pnp: PnP ACPI: disabled869worker2 # [ 0.213725] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)870worker2 # [ 0.213761] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)871worker1 # [ 0.181452] NET: Registered PF_INET protocol family872worker2 # [ 0.213854] NET: Registered PF_UNIX/PF_LOCAL protocol family873worker1 # [ 0.181713] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)874worker2 # [ 0.213875] NET: Registered PF_XDP protocol family875worker2 # [ 0.213892] PCI: CLS 0 bytes, default 64876client # [ 0.181821] NET: Registered PF_INET protocol family877worker2 # [ 0.214150] Trying to unpack rootfs image as initramfs...878client # [ 0.182093] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)879worker2 # [ 0.228365] kvm [1]: HYP mode not available880lb # [ 0.210745] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881lb # [ 0.210787] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882lb # [ 0.210812] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883lb # [ 0.210879] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884lb # [ 0.211022] TCP: Hash tables configured (established 16384 bind 16384)885lb # [ 0.211105] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)886lb # [ 0.211149] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)887lb # [ 0.211184] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)888lb # [ 0.211299] NET: Registered PF_UNIX/PF_LOCAL protocol family889lb # [ 0.211319] NET: Registered PF_XDP protocol family890lb # [ 0.211335] PCI: CLS 0 bytes, default 64891lb # [ 0.211580] Trying to unpack rootfs image as initramfs...892lb # [ 0.226158] kvm [1]: HYP mode not available893worker1 # [ 0.211115] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)894worker1 # [ 0.211160] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)895worker1 # [ 0.211189] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)896worker1 # [ 0.211265] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)897worker1 # [ 0.211408] TCP: Hash tables configured (established 16384 bind 16384)898worker1 # [ 0.211500] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)899worker1 # [ 0.211536] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker1 # [ 0.211573] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)901client # [ 0.212723] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)902client # [ 0.212773] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)903client # [ 0.212805] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)904client # [ 0.212889] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)905client # [ 0.213032] TCP: Hash tables configured (established 16384 bind 16384)906client # [ 0.213118] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)907client # [ 0.213166] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)908worker1 # [ 0.219048] NET: Registered PF_UNIX/PF_LOCAL protocol family909worker1 # [ 0.219085] NET: Registered PF_XDP protocol family910client # [ 0.213234] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)911worker1 # [ 0.219105] PCI: CLS 0 bytes, default 64912client # [ 0.213326] NET: Registered PF_UNIX/PF_LOCAL protocol family913worker1 # [ 0.219341] Trying to unpack rootfs image as initramfs...914client # [ 0.213346] NET: Registered PF_XDP protocol family915client # [ 0.213364] PCI: CLS 0 bytes, default 64916client # [ 0.213620] Trying to unpack rootfs image as initramfs...917worker1 # [ 0.230173] kvm [1]: HYP mode not available918client # [ 0.228138] kvm [1]: HYP mode not available919worker2 # [ 0.324432] Initialise system trusted keyrings920worker2 # [ 0.325170] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker2 # [ 0.326409] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker2 # [ 0.327223] 9p: Installing v9fs 9p2000 file system support923lb # [ 0.320956] Initialise system trusted keyrings924lb # [ 0.321705] workingset: timestamp_bits=42 max_order=19 bucket_order=0925lb # [ 0.323024] squashfs: version 4.0 (2009/01/31) Phillip Lougher926lb # [ 0.323790] 9p: Installing v9fs 9p2000 file system support927worker2 # [ 0.355272] Key type asymmetric registered928worker2 # [ 0.355298] Asymmetric key parser 'x509' registered929worker2 # [ 0.355385] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)930worker2 # [ 0.357485] io scheduler mq-deadline registered931worker2 # [ 0.357497] io scheduler kyber registered932worker1 # [ 0.324198] Initialise system trusted keyrings933worker1 # [ 0.324979] workingset: timestamp_bits=42 max_order=19 bucket_order=0934worker2 # [ 0.366940] pl061_gpio 9030000.pl061: PL061 GPIO chip registered935worker1 # [ 0.326262] squashfs: version 4.0 (2009/01/31) Phillip Lougher936client # [ 0.323745] Initialise system trusted keyrings937worker1 # [ 0.327063] 9p: Installing v9fs 9p2000 file system support938client # [ 0.324492] workingset: timestamp_bits=42 max_order=19 bucket_order=0939worker2 # [ 0.368360] ledtrig-cpu: registered to indicate activity on CPUs940client # [ 0.325713] squashfs: version 4.0 (2009/01/31) Phillip Lougher941worker2 # [ 0.368747] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:942lb # [ 0.352466] Key type asymmetric registered943worker2 # [ 0.368765] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000944lb # [ 0.352493] Asymmetric key parser 'x509' registered945lb # [ 0.352570] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)946worker2 # [ 0.368777] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000947lb # [ 0.354701] io scheduler mq-deadline registered948worker2 # [ 0.368786] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000949lb # [ 0.354713] io scheduler kyber registered950client # [ 0.326561] 9p: Installing v9fs 9p2000 file system support951worker2 # [ 0.368806] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits952worker2 # [ 0.368832] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]953worker2 # [ 0.368908] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00954worker2 # [ 0.368917] pci_bus 0000:00: root bus resource [bus 00-ff]955worker2 # [ 0.368923] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]956worker2 # [ 0.368928] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]957worker2 # [ 0.368933] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]958worker2 # [ 0.368990] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint959worker2 # [ 0.369433] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint960worker2 # [ 0.369619] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]961worker2 # [ 0.369636] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]962worker2 # [ 0.369666] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]963lb # [ 0.364518] pl061_gpio 9030000.pl061: PL061 GPIO chip registered964worker2 # [ 0.369682] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]965worker2 # [ 0.370303] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint966worker2 # [ 0.370485] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]967worker2 # [ 0.370501] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]968lb # [ 0.365937] ledtrig-cpu: registered to indicate activity on CPUs969worker2 # [ 0.370530] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]970lb # [ 0.366334] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:971lb # [ 0.366351] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000972lb # [ 0.366363] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000973lb # [ 0.366371] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000974worker1 # [ 0.351106] Key type asymmetric registered975lb # [ 0.366393] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits976worker1 # [ 0.351131] Asymmetric key parser 'x509' registered977worker1 # [ 0.351207] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)978lb # [ 0.366425] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]979lb # [ 0.366511] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00980lb # [ 0.366520] pci_bus 0000:00: root bus resource [bus 00-ff]981lb # [ 0.366526] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]982lb # [ 0.366532] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]983lb # [ 0.366537] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]984lb # [ 0.366627] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint985client # [ 0.354632] Key type asymmetric registered986client # [ 0.354659] Asymmetric key parser 'x509' registered987lb # [ 0.367072] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint988lb # [ 0.367260] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]989client # [ 0.354727] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)990lb # [ 0.367277] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]991client # [ 0.356922] io scheduler mq-deadline registered992client # [ 0.356933] io scheduler kyber registered993lb # [ 0.367307] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker2 # [ 0.395097] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint995lb # [ 0.367323] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]996worker2 # [ 0.395304] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]997worker2 # [ 0.395321] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]998lb # [ 0.367782] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint999worker2 # [ 0.395352] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1000lb # [ 0.367964] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1001worker1 # [ 0.359714] io scheduler mq-deadline registered1002worker1 # [ 0.359733] io scheduler kyber registered1003lb # [ 0.367979] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1004worker2 # [ 0.395808] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1005lb # [ 0.368009] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006worker2 # [ 0.395990] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1007worker2 # [ 0.396006] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1008worker1 # [ 0.364899] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1009worker2 # [ 0.396035] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1010lb # [ 0.387681] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1011worker2 # [ 0.396054] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1012lb # [ 0.387867] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1013worker1 # [ 0.366411] ledtrig-cpu: registered to indicate activity on CPUs1014lb # [ 0.387883] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1015worker2 # [ 0.396531] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1016worker1 # [ 0.366797] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1017lb # [ 0.387913] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1018worker2 # [ 0.396718] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1019worker1 # [ 0.366814] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001020worker2 # [ 0.396748] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1021worker2 # [ 0.397200] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1022client # [ 0.366391] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1023worker1 # [ 0.366834] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001024worker2 # [ 0.397387] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1025worker1 # [ 0.366843] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001026worker2 # [ 0.397417] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1027worker1 # [ 0.366868] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1028worker2 # [ 0.397849] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1029worker2 # [ 0.398031] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1030client # [ 0.367836] ledtrig-cpu: registered to indicate activity on CPUs1031worker1 # [ 0.366890] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1032client # [ 0.368237] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1033worker1 # [ 0.366969] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001034worker2 # [ 0.398286] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1035worker1 # [ 0.366979] pci_bus 0000:00: root bus resource [bus 00-ff]1036worker2 # [ 0.398473] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1037client # [ 0.368254] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001038worker1 # [ 0.366985] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1039worker2 # [ 0.398503] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1040client # [ 0.368266] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001041worker1 # [ 0.366990] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1042worker2 # [ 0.398962] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1043worker1 # [ 0.366996] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1044worker2 # [ 0.399147] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1045client # [ 0.368275] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001046worker2 # [ 0.399176] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1047worker1 # [ 0.367097] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1048client # [ 0.368295] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1049worker2 # [ 0.399622] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1050worker1 # [ 0.367534] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1051client # [ 0.368319] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1052worker2 # [ 0.399807] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1053client # [ 0.368399] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001054worker2 # [ 0.399844] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1055client # [ 0.368408] pci_bus 0000:00: root bus resource [bus 00-ff]1056worker2 # [ 0.400305] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1057client # [ 0.368414] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1058lb # [ 0.396448] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1059worker2 # [ 0.400586] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1060client # [ 0.368419] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1061lb # [ 0.396655] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1062worker2 # [ 0.400604] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1063lb # [ 0.396672] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1064client # [ 0.368424] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1065worker2 # [ 0.400633] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1066lb # [ 0.396701] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067client # [ 0.368522] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1068worker2 # [ 0.401085] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1069lb # [ 0.396721] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1070worker2 # [ 0.401266] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1071client # [ 0.368961] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1072lb # [ 0.397248] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1073worker2 # [ 0.401282] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1074client # [ 0.369146] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1075lb # [ 0.397435] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1076worker2 # [ 0.401312] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077client # [ 0.369163] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1078lb # [ 0.397465] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1079worker2 # [ 0.401874] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1080client # [ 0.369193] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081client # [ 0.369210] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1082worker2 # [ 0.401885] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1083lb # [ 0.397927] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1084lb # [ 0.398123] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1085client # [ 0.369664] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1086worker2 # [ 0.401891] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1087client # [ 0.369847] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1088lb # [ 0.398152] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089worker2 # [ 0.401935] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1090client # [ 0.369863] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1091lb # [ 0.398555] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1092worker2 # [ 0.401982] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1093client # [ 0.369893] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094lb # [ 0.398737] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1095worker2 # [ 0.402031] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1096lb # [ 0.399023] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1097worker2 # [ 0.402076] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1098lb # [ 0.399210] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1099lb # [ 0.399239] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100worker2 # [ 0.402122] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1101worker1 # [ 0.387800] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1102worker2 # [ 0.402168] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1103lb # [ 0.399698] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1104worker1 # [ 0.387832] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1105lb # [ 0.399886] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1106worker2 # [ 0.402213] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1107worker1 # [ 0.387862] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108lb # [ 0.399916] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109worker2 # [ 0.402259] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1110worker1 # [ 0.387878] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1111lb # [ 0.400411] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1112worker2 # [ 0.402304] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1113worker1 # [ 0.388398] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1114lb # [ 0.400597] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1115worker2 # [ 0.402378] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1116worker1 # [ 0.388588] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1117lb # [ 0.400627] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118worker1 # [ 0.388603] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1119worker2 # [ 0.402424] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1120lb # [ 0.401096] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1121worker2 # [ 0.402445] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1122worker1 # [ 0.388632] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123lb # [ 0.401385] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1124worker2 # [ 0.402467] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1125lb # [ 0.401402] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1126worker1 # [ 0.389093] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1127worker1 # [ 0.389276] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1128worker2 # [ 0.402488] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1129lb # [ 0.401440] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1130worker1 # [ 0.389292] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1131worker2 # [ 0.402509] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1132lb # [ 0.401903] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1133worker1 # [ 0.389322] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1134worker2 # [ 0.402530] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1135lb # [ 0.402088] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1136worker2 # [ 0.402551] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1137lb # [ 0.402106] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1138worker1 # [ 0.389775] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1139worker1 # [ 0.389963] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1140worker2 # [ 0.402577] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1141lb # [ 0.402140] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1142client # [ 0.394414] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1143worker1 # [ 0.389978] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1144worker2 # [ 0.402598] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1145lb # [ 0.402711] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1146client # [ 0.394626] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1147worker1 # [ 0.390008] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148worker2 # [ 0.402620] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1149client # [ 0.394643] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1150lb # [ 0.402722] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1151worker1 # [ 0.390026] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1152worker2 # [ 0.402643] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1153client # [ 0.394674] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1154lb # [ 0.402727] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1155worker1 # [ 0.390507] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1156worker2 # [ 0.402664] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1157client # [ 0.395193] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1158lb # [ 0.402772] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1159worker1 # [ 0.390696] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1160worker2 # [ 0.402686] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1161client # [ 0.395381] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1162lb # [ 0.402819] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1163worker2 # [ 0.402707] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1164worker1 # [ 0.390726] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1165client # [ 0.395397] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1166worker2 # [ 0.402727] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1167lb # [ 0.402866] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1168client # [ 0.395428] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1169worker1 # [ 0.391178] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1170worker2 # [ 0.402748] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1171worker1 # [ 0.391366] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1172client # [ 0.395447] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1173lb # [ 0.402913] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1174worker2 # [ 0.402769] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1175worker1 # [ 0.391395] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176client # [ 0.395929] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1177lb # [ 0.402960] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1178client # [ 0.396116] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1179worker1 # [ 0.391818] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1180lb # [ 0.403006] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1181client # [ 0.396146] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1182worker1 # [ 0.392002] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1183lb # [ 0.403052] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1184client # [ 0.396602] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1185worker1 # [ 0.392279] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1186lb # [ 0.403097] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1187client # [ 0.396791] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1188worker1 # [ 0.392467] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1189lb # [ 0.403144] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1190client # [ 0.396820] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1191worker1 # [ 0.392497] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1192lb # [ 0.403212] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1193client # [ 0.397207] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1194worker1 # [ 0.392956] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1195lb # [ 0.403257] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1196client # [ 0.397390] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1197worker1 # [ 0.393143] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1198lb # [ 0.403279] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1199client # [ 0.397670] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1200worker1 # [ 0.393172] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1201worker2 # [ 0.458855] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1202lb # [ 0.403301] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1203client # [ 0.397856] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1204worker1 # [ 0.393623] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1205worker2 # [ 0.458905] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1206lb # [ 0.403322] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1207client # [ 0.397886] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1208worker1 # [ 0.393810] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1209worker2 # [ 0.458915] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1210lb # [ 0.403344] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1211worker1 # [ 0.393839] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1212worker2 # [ 0.458920] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1213client # [ 0.398353] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1214lb # [ 0.403365] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1215worker2 # [ 0.459777] pci 0000:00:07.0: enabling device (0000 -> 0002)1216client # [ 0.398546] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1217worker1 # [ 0.394309] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1218lb # [ 0.403387] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1219worker1 # [ 0.394592] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1220client # [ 0.398580] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1221lb # [ 0.403412] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1222worker1 # [ 0.394610] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1223client # [ 0.399068] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1224lb # [ 0.403433] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1225worker1 # [ 0.394640] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1226client # [ 0.399255] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1227lb # [ 0.403455] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1228worker1 # [ 0.395100] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1229client # [ 0.399295] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1230lb # [ 0.403477] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1231worker1 # [ 0.395283] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1232client # [ 0.399752] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1233lb # [ 0.403499] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1234worker1 # [ 0.395298] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1235client # [ 0.400035] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1236lb # [ 0.403521] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1237worker1 # [ 0.395327] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1238client # [ 0.400052] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1239lb # [ 0.403542] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1240client # [ 0.400083] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1241lb # [ 0.403563] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1242lb # [ 0.403584] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1243client # [ 0.400544] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1244lb # [ 0.403605] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1245client # [ 0.400728] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1246lb # [ 0.403626] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1247client # [ 0.400744] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1248lb # [ 0.403652] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1249client # [ 0.400774] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1250lb # [ 0.403661] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1251client # [ 0.401350] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1252lb # [ 0.403665] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1253client # [ 0.401362] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1254client # [ 0.401367] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1255client # [ 0.401412] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1256client # [ 0.401460] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1257client # [ 0.401507] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1258client # [ 0.401555] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1259worker1 # [ 0.436051] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1260client # [ 0.401602] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1261lb # [ 0.460587] pci 0000:00:07.0: enabling device (0000 -> 0002)1262worker1 # [ 0.436073] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1263client # [ 0.401649] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1264worker1 # [ 0.436078] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1265client # [ 0.401696] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1266worker1 # [ 0.436131] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1267client # [ 0.401742] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1268worker1 # [ 0.436178] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1269client # [ 0.401789] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1270worker1 # [ 0.436224] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1271client # [ 0.401858] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1272worker1 # [ 0.436272] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1273client # [ 0.401904] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1274client # [ 0.401926] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1275worker1 # [ 0.436319] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1276client # [ 0.401948] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1277worker1 # [ 0.436367] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1278client # [ 0.401970] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1279worker1 # [ 0.436414] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1280client # [ 0.401991] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1281worker2 # [ 0.489154] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1282worker1 # [ 0.436461] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1283client # [ 0.402013] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1284worker1 # [ 0.436507] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1285client # [ 0.402035] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1286client # [ 0.402060] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1287worker1 # [ 0.436631] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1288client # [ 0.402082] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1289worker1 # [ 0.436679] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1290client # [ 0.402104] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1291worker1 # [ 0.436700] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1292client # [ 0.402127] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1293worker1 # [ 0.436721] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1294client # [ 0.402149] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1295worker1 # [ 0.436742] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1296client # [ 0.402171] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1297worker1 # [ 0.436763] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1298client # [ 0.402192] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1299worker1 # [ 0.436784] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1300client # [ 0.402214] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1301worker1 # [ 0.436805] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1302client # [ 0.402235] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1303worker1 # [ 0.436830] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1304worker1 # [ 0.436852] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1305worker1 # [ 0.436874] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1306worker1 # [ 0.436895] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1307worker1 # [ 0.436917] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1308worker1 # [ 0.436938] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1309worker1 # [ 0.436958] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1310worker1 # [ 0.436979] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1311worker1 # [ 0.437000] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1312worker2 # [ 0.499017] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1313worker1 # [ 0.437020] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1314worker2 # [ 0.502373] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315worker1 # [ 0.437041] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1316lb # [ 0.486639] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1317worker2 # [ 0.504513] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1318worker1 # [ 0.437066] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1319worker2 # [ 0.506480] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1320worker1 # [ 0.437075] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1321worker1 # [ 0.437080] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1322worker1 # [ 0.437900] pci 0000:00:07.0: enabling device (0000 -> 0002)1323client # [ 0.466352] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1324client # [ 0.466397] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1325client # [ 0.466435] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1326client # [ 0.466445] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1327lb # [ 0.492628] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1328client # [ 0.466450] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1329lb # [ 0.494668] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1330client # [ 0.467281] pci 0000:00:07.0: enabling device (0000 -> 0002)1331worker2 # [ 0.516843] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1332worker2 # [ 0.518648] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1333lb # [ 0.504597] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1334lb # [ 0.506643] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1335worker2 # [ 0.529208] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1336worker1 # [ 0.488880] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1337worker1 # [ 0.491041] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1338client # [ 0.488446] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1339worker2 # [ 0.535795] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1340worker2 # [ 0.537702] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1341lb # [ 0.516534] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1342lb # [ 0.519038] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1343worker1 # [ 0.502143] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1344lb # [ 0.520876] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1345lb # [ 0.522668] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1346client # [ 0.498490] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1347client # [ 0.501799] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1348worker2 # [ 0.548309] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1349worker1 # [ 0.508209] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1350worker1 # [ 0.510224] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1351lb # [ 0.532545] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1352lb # [ 0.534859] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1353worker2 # [ 0.557650] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1354client # [ 0.511604] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1355worker1 # [ 0.519843] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1356worker2 # [ 0.564037] msm_serial: driver initialized1357worker1 # [ 0.521597] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1358worker2 # [ 0.564215] SuperH (H)SCI(F) driver initialized1359worker1 # [ 0.523558] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1360worker2 # [ 0.564269] STM32 USART driver initialized1361lb # [ 0.552465] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1362client # [ 0.522869] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1363lb # [ 0.555103] msm_serial: driver initialized1364lb # [ 0.555257] SuperH (H)SCI(F) driver initialized1365client # [ 0.525046] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1366lb # [ 0.555313] STM32 USART driver initialized1367client # [ 0.526876] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1368worker1 # [ 0.533590] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1369client # [ 0.528813] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1370worker1 # [ 0.535576] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1371client # [ 0.538868] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1372client # [ 0.540768] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1373worker1 # [ 0.545764] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1374worker2 # [ 0.595674] loop: module loaded1375worker2 # [ 0.595859] virtio_blk virtio2: 1/0/0 default/read/poll queues1376worker2 # [ 0.596806] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1377client # [ 0.550878] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1378worker1 # [ 0.559710] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1379lb # [ 0.587290] loop: module loaded1380lb # [ 0.587500] virtio_blk virtio2: 1/0/0 default/read/poll queues1381lb # [ 0.588482] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1382client # [ 0.566407] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1383worker1 # [ 0.568728] msm_serial: driver initialized1384worker1 # [ 0.568878] SuperH (H)SCI(F) driver initialized1385worker1 # [ 0.568933] STM32 USART driver initialized1386worker2 # [ 0.607412] megasas: 07.734.00.00-rc11387client # [ 0.568952] msm_serial: driver initialized1388worker2 # [ 0.608184] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1389client # [ 0.569102] SuperH (H)SCI(F) driver initialized1390lb # [ 0.592936] megasas: 07.734.00.00-rc11391client # [ 0.569155] STM32 USART driver initialized1392worker2 # [ 0.610423] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001393lb # [ 0.593636] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1394worker2 # [ 0.610447] Intel/Sharp Extended Query Table at 0x00311395worker2 # [ 0.611975] Using buffer write method1396worker2 # [ 0.612050] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1397worker2 # [ 0.613768] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001398worker2 # [ 0.613791] Intel/Sharp Extended Query Table at 0x00311399worker2 # [ 0.627358] Using buffer write method1400worker2 # [ 0.627392] Concatenating MTD devices:1401worker2 # [ 0.627396] (0): "0.flash"1402worker2 # [ 0.627400] (1): "0.flash"1403worker2 # [ 0.627409] into device "0.flash"1404lb # [ 0.605414] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001405lb # [ 0.605445] Intel/Sharp Extended Query Table at 0x00311406lb # [ 0.606987] Using buffer write method1407lb # [ 0.607065] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1408worker1 # [ 0.598476] loop: module loaded1409worker1 # [ 0.598661] virtio_blk virtio2: 1/0/0 default/read/poll queues1410worker1 # [ 0.599473] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1411client # [ 0.597239] loop: module loaded1412client # [ 0.597409] virtio_blk virtio2: 1/0/0 default/read/poll queues1413lb # [ 0.622464] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001414client # [ 0.598180] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1415lb # [ 0.622494] Intel/Sharp Extended Query Table at 0x00311416lb # [ 0.626432] Using buffer write method1417lb # [ 0.626456] Concatenating MTD devices:1418lb # [ 0.626460] (0): "0.flash"1419lb # [ 0.626464] (1): "0.flash"1420lb # [ 0.626475] into device "0.flash"1421worker1 # [ 0.604260] megasas: 07.734.00.00-rc11422worker1 # [ 0.604944] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1423worker1 # [ 0.616176] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001424worker1 # [ 0.616209] Intel/Sharp Extended Query Table at 0x00311425worker1 # [ 0.617847] Using buffer write method1426client # [ 0.610981] megasas: 07.734.00.00-rc11427worker1 # [ 0.617927] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1428client # [ 0.611744] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1429client # [ 0.613996] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001430client # [ 0.614023] Intel/Sharp Extended Query Table at 0x00311431client # [ 0.615647] Using buffer write method1432client # [ 0.615720] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1433client # [ 0.617431] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001434client # [ 0.617455] Intel/Sharp Extended Query Table at 0x00311435worker1 # [ 0.632415] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001436worker1 # [ 0.632446] Intel/Sharp Extended Query Table at 0x00311437worker1 # [ 0.634140] Using buffer write method1438worker1 # [ 0.634168] Concatenating MTD devices:1439worker1 # [ 0.634172] (0): "0.flash"1440worker1 # [ 0.634176] (1): "0.flash"1441worker1 # [ 0.634187] into device "0.flash"1442client # [ 0.634877] Using buffer write method1443client # [ 0.634914] Concatenating MTD devices:1444client # [ 0.634918] (0): "0.flash"1445client # [ 0.634922] (1): "0.flash"1446client # [ 0.634932] into device "0.flash"1447worker2 # [ 0.888168] Freeing initrd memory: 26944K1448worker2 # [ 0.894180] tun: Universal TUN/TAP device driver, 1.61449worker2 # [ 0.898025] thunder_xcv, ver 1.01450worker2 # [ 0.898073] thunder_bgx, ver 1.01451lb # [ 0.877907] Freeing initrd memory: 26948K1452worker2 # [ 0.898096] nicpf, ver 1.01453worker2 # [ 0.898665] e1000: Intel(R) PRO/1000 Network Driver1454worker2 # [ 0.898673] e1000: Copyright (c) 1999-2006 Intel Corporation.1455worker2 # [ 0.898697] e1000e: Intel(R) PRO/1000 Network Driver1456worker2 # [ 0.898705] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457worker2 # [ 0.898733] igb: Intel(R) Gigabit Ethernet Network Driver1458worker2 # [ 0.898739] igb: Copyright (c) 2007-2014 Intel Corporation.1459lb # [ 0.883881] tun: Universal TUN/TAP device driver, 1.61460worker2 # [ 0.898761] igbvf: Intel(R) Gigabit Virtual Function Network Driver1461worker2 # [ 0.898766] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1462worker2 # [ 0.905486] sky2: driver version 1.301463lb # [ 0.887692] thunder_xcv, ver 1.01464worker2 # [ 0.907466] ehci-pci 0000:00:07.0: EHCI Host Controller1465lb # [ 0.887736] thunder_bgx, ver 1.01466lb # [ 0.887758] nicpf, ver 1.01467worker2 # [ 0.907498] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11468lb # [ 0.888332] e1000: Intel(R) PRO/1000 Network Driver1469worker2 # [ 0.907783] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001470lb # [ 0.888339] e1000: Copyright (c) 1999-2006 Intel Corporation.1471lb # [ 0.888362] e1000e: Intel(R) PRO/1000 Network Driver1472lb # [ 0.888371] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1473lb # [ 0.892171] igb: Intel(R) Gigabit Ethernet Network Driver1474worker2 # [ 0.910888] usbcore: registered new interface driver usb-storage1475lb # [ 0.892182] igb: Copyright (c) 2007-2014 Intel Corporation.1476worker2 # [ 0.910979] usbcore: registered new interface driver usbserial_generic1477lb # [ 0.892231] igbvf: Intel(R) Gigabit Virtual Function Network Driver1478worker2 # [ 0.910993] usbserial: USB Serial support registered for generic1479lb # [ 0.892237] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1480worker2 # [ 0.911583] hv_vmbus: registering driver hyperv_keyboard1481lb # [ 0.895293] sky2: driver version 1.301482worker2 # [ 0.915373] rtc-pl031 9010000.pl031: registered as rtc01483worker2 # [ 0.915398] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:58:26 UTC (1790074706)1484worker2 # [ 0.915699] i2c_dev: i2c /dev entries driver1485lb # [ 0.897357] usbcore: registered new interface driver usb-storage1486lb # [ 0.897411] usbcore: registered new interface driver usbserial_generic1487lb # [ 0.897424] usbserial: USB Serial support registered for generic1488lb # [ 0.898023] hv_vmbus: registering driver hyperv_keyboard1489lb # [ 0.899240] ehci-pci 0000:00:07.0: EHCI Host Controller1490lb # [ 0.899265] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11491worker2 # [ 0.921093] sdhci: Secure Digital Host Controller Interface driver1492lb # [ 0.899498] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001493worker2 # [ 0.921103] sdhci: Copyright(c) Pierre Ossman1494worker2 # [ 0.921366] Synopsys Designware Multimedia Card Interface Driver1495worker2 # [ 0.921735] sdhci-pltfm: SDHCI platform and OF driver helper1496lb # [ 0.905024] rtc-pl031 9010000.pl031: registered as rtc01497worker2 # [ 0.922818] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001498worker2 # [ 0.923132] hub 1-0:1.0: USB hub found1499lb # [ 0.905050] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:58:26 UTC (1790074706)1500worker2 # [ 0.923151] hub 1-0:1.0: 6 ports detected1501lb # [ 0.905385] i2c_dev: i2c /dev entries driver1502worker2 # [ 0.928324] hid: raw HID events driver (C) Jiri Kosina1503worker2 # [ 0.928564] usbcore: registered new interface driver usbhid1504worker2 # [ 0.928571] usbhid: USB HID core driver1505client # [ 0.884912] Freeing initrd memory: 26972K1506worker1 # [ 0.890615] Freeing initrd memory: 26976K1507lb # [ 0.910714] sdhci: Secure Digital Host Controller Interface driver1508lb # [ 0.910726] sdhci: Copyright(c) Pierre Ossman1509lb # [ 0.910993] Synopsys Designware Multimedia Card Interface Driver1510worker2 # [ 0.931527] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1511lb # [ 0.911362] sdhci-pltfm: SDHCI platform and OF driver helper1512worker2 # [ 0.933070] drop_monitor: Initializing network drop monitor service1513lb # [ 0.915110] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001514worker2 # [ 0.933218] NET: Registered PF_INET6 protocol family1515worker2 # [ 0.936268] Segment Routing with IPv61516client # [ 0.891065] tun: Universal TUN/TAP device driver, 1.61517worker2 # [ 0.936298] In-situ OAM (IOAM) with IPv61518worker2 # [ 0.936326] NET: Registered PF_PACKET protocol family1519lb # [ 0.916475] hid: raw HID events driver (C) Jiri Kosina1520lb # [ 0.916747] usbcore: registered new interface driver usbhid1521worker1 # [ 0.896714] tun: Universal TUN/TAP device driver, 1.61522worker2 # [ 0.937940] 9pnet: Installing 9P2000 support1523lb # [ 0.916754] usbhid: USB HID core driver1524worker2 # [ 0.937989] Key type dns_resolver registered1525lb # [ 0.919854] hub 1-0:1.0: USB hub found1526lb # [ 0.920412] hub 1-0:1.0: 6 ports detected1527lb # [ 0.921365] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1528lb # [ 0.922898] drop_monitor: Initializing network drop monitor service1529client # [ 0.894747] thunder_xcv, ver 1.01530client # [ 0.894799] thunder_bgx, ver 1.01531lb # [ 0.923075] NET: Registered PF_INET6 protocol family1532client # [ 0.894822] nicpf, ver 1.01533client # [ 0.895416] e1000: Intel(R) PRO/1000 Network Driver1534lb # [ 0.926218] Segment Routing with IPv61535worker1 # [ 0.900438] thunder_xcv, ver 1.01536lb # [ 0.926237] In-situ OAM (IOAM) with IPv61537client # [ 0.895425] e1000: Copyright (c) 1999-2006 Intel Corporation.1538worker1 # [ 0.900484] thunder_bgx, ver 1.01539worker2 # [ 0.944781] registered taskstats version 11540lb # [ 0.926263] NET: Registered PF_PACKET protocol family1541worker1 # [ 0.900506] nicpf, ver 1.01542client # [ 0.895448] e1000e: Intel(R) PRO/1000 Network Driver1543worker2 # [ 0.944934] Loading compiled-in X.509 certificates1544worker1 # [ 0.901082] e1000: Intel(R) PRO/1000 Network Driver1545client # [ 0.895457] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1546lb # [ 0.927899] 9pnet: Installing 9P2000 support1547worker1 # [ 0.901090] e1000: Copyright (c) 1999-2006 Intel Corporation.1548client # [ 0.895487] igb: Intel(R) Gigabit Ethernet Network Driver1549lb # [ 0.927943] Key type dns_resolver registered1550worker1 # [ 0.901112] e1000e: Intel(R) PRO/1000 Network Driver1551client # [ 0.895493] igb: Copyright (c) 2007-2014 Intel Corporation.1552worker1 # [ 0.901120] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1553client # [ 0.895517] igbvf: Intel(R) Gigabit Virtual Function Network Driver1554worker1 # [ 0.901148] igb: Intel(R) Gigabit Ethernet Network Driver1555client # [ 0.895523] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1556worker1 # [ 0.901154] igb: Copyright (c) 2007-2014 Intel Corporation.1557client # [ 0.895658] sky2: driver version 1.301558worker1 # [ 0.901177] igbvf: Intel(R) Gigabit Virtual Function Network Driver1559client # [ 0.897250] usbcore: registered new interface driver usb-storage1560worker1 # [ 0.901183] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1561client # [ 0.897372] usbcore: registered new interface driver usbserial_generic1562worker1 # [ 0.901319] sky2: driver version 1.301563client # [ 0.897387] usbserial: USB Serial support registered for generic1564worker1 # [ 0.902934] usbcore: registered new interface driver usb-storage1565client # [ 0.897989] hv_vmbus: registering driver hyperv_keyboard1566worker2 # [ 0.953904] Demotion targets for Node 0: null1567worker1 # [ 0.903025] usbcore: registered new interface driver usbserial_generic1568worker2 # [ 0.954023] Key type .fscrypt registered1569client # [ 0.899090] ehci-pci 0000:00:07.0: EHCI Host Controller1570worker1 # [ 0.903039] usbserial: USB Serial support registered for generic1571worker2 # [ 0.954030] Key type fscrypt-provisioning registered1572client # [ 0.899120] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11573worker1 # [ 0.903638] hv_vmbus: registering driver hyperv_keyboard1574worker2 # [ 0.954127] ima: No TPM chip found, activating TPM-bypass!1575lb # [ 0.934536] registered taskstats version 11576worker2 # [ 0.954146] ima: Allocated hash algorithm: sha11577worker1 # [ 0.904376] ehci-pci 0000:00:07.0: EHCI Host Controller1578lb # [ 0.934716] Loading compiled-in X.509 certificates1579worker2 # [ 0.954168] ima: No architecture policies found1580client # [ 0.899403] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001581worker1 # [ 0.904402] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11582client # [ 0.911052] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001583client # [ 0.911361] hub 1-0:1.0: USB hub found1584worker2 # [ 0.958076] input: gpio-keys as /devices/platform/gpio-keys/input/input01585client # [ 0.911377] hub 1-0:1.0: 6 ports detected1586worker1 # [ 0.904716] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001587worker1 # [ 0.916301] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001588worker1 # [ 0.916596] hub 1-0:1.0: USB hub found1589worker1 # [ 0.916610] hub 1-0:1.0: 6 ports detected1590client # [ 0.914776] rtc-pl031 9010000.pl031: registered as rtc01591client # [ 0.914806] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:58:26 UTC (1790074706)1592client # [ 0.915127] i2c_dev: i2c /dev entries driver1593worker1 # [ 0.920310] rtc-pl031 9010000.pl031: registered as rtc01594lb # [ 0.943477] Demotion targets for Node 0: null1595lb # [ 0.943583] Key type .fscrypt registered1596worker1 # [ 0.920337] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:58:26 UTC (1790074706)1597worker1 # [ 0.920638] i2c_dev: i2c /dev entries driver1598lb # [ 0.943590] Key type fscrypt-provisioning registered1599lb # [ 0.943688] ima: No TPM chip found, activating TPM-bypass!1600lb # [ 0.943707] ima: Allocated hash algorithm: sha11601lb # [ 0.943727] ima: No architecture policies found1602client # [ 0.920572] sdhci: Secure Digital Host Controller Interface driver1603lb # [ 0.947956] input: gpio-keys as /devices/platform/gpio-keys/input/input01604client # [ 0.920582] sdhci: Copyright(c) Pierre Ossman1605client # [ 0.920851] Synopsys Designware Multimedia Card Interface Driver1606worker1 # [ 0.925881] sdhci: Secure Digital Host Controller Interface driver1607client # [ 0.921237] sdhci-pltfm: SDHCI platform and OF driver helper1608worker1 # [ 0.925892] sdhci: Copyright(c) Pierre Ossman1609worker1 # [ 0.926158] Synopsys Designware Multimedia Card Interface Driver1610worker1 # [ 0.926539] sdhci-pltfm: SDHCI platform and OF driver helper1611client # [ 0.925514] hid: raw HID events driver (C) Jiri Kosina1612client # [ 0.925765] usbcore: registered new interface driver usbhid1613client # [ 0.925771] usbhid: USB HID core driver1614worker1 # [ 0.930800] hid: raw HID events driver (C) Jiri Kosina1615worker1 # [ 0.931063] usbcore: registered new interface driver usbhid1616worker1 # [ 0.931069] usbhid: USB HID core driver1617client # [ 0.928704] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1618client # [ 0.931389] drop_monitor: Initializing network drop monitor service1619client # [ 0.931535] NET: Registered PF_INET6 protocol family1620worker1 # [ 0.933964] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1621worker2 # [ 0.976717] clk: Disabling unused clocks1622worker1 # [ 0.935532] drop_monitor: Initializing network drop monitor service1623client # [ 0.933488] Segment Routing with IPv61624client # [ 0.933518] In-situ OAM (IOAM) with IPv61625worker2 # [ 0.976743] PM: genpd: Disabling unused power domains1626worker1 # [ 0.937566] NET: Registered PF_INET6 protocol family1627client # [ 0.933546] NET: Registered PF_PACKET protocol family1628worker1 # [ 0.938735] Segment Routing with IPv61629worker1 # [ 0.938755] In-situ OAM (IOAM) with IPv61630worker2 # [ 0.980899] Freeing unused kernel memory: 4736K1631worker1 # [ 0.938783] NET: Registered PF_PACKET protocol family1632worker2 # [ 0.981171] Run /init as init process1633client # [ 0.935382] 9pnet: Installing 9P2000 support1634client # [ 0.935429] Key type dns_resolver registered1635worker1 # [ 0.940630] 9pnet: Installing 9P2000 support1636worker1 # [ 0.940678] Key type dns_resolver registered1637lb # [ 0.966041] clk: Disabling unused clocks1638lb # [ 0.966071] PM: genpd: Disabling unused power domains1639worker1 # [ 0.947201] registered taskstats version 11640worker1 # [ 0.947345] Loading compiled-in X.509 certificates1641lb # [ 0.970253] Freeing unused kernel memory: 4736K1642lb # [ 0.970493] Run /init as init process1643client # [ 0.942314] registered taskstats version 11644client # [ 0.942479] Loading compiled-in X.509 certificates1645client # [ 0.951115] Demotion targets for Node 0: null1646worker2 # [ 0.997078] systemd[1]: Successfully made /usr/ read-only.1647client # [ 0.951222] Key type .fscrypt registered1648client # [ 0.951229] Key type fscrypt-provisioning registered1649worker1 # [ 0.956100] Demotion targets for Node 0: null1650client # [ 0.951330] ima: No TPM chip found, activating TPM-bypass!1651worker1 # [ 0.956198] Key type .fscrypt registered1652client # [ 0.951349] ima: Allocated hash algorithm: sha11653worker1 # [ 0.956205] Key type fscrypt-provisioning registered1654client # [ 0.951372] ima: No architecture policies found1655worker1 # [ 0.956299] ima: No TPM chip found, activating TPM-bypass!1656worker1 # [ 0.956318] ima: Allocated hash algorithm: sha11657worker1 # [ 0.956340] ima: No architecture policies found1658client # [ 0.955664] input: gpio-keys as /devices/platform/gpio-keys/input/input01659worker1 # [ 0.960686] input: gpio-keys as /devices/platform/gpio-keys/input/input01660lb # [ 0.986625] systemd[1]: Successfully made /usr/ read-only.1661worker1 # [ 0.979218] clk: Disabling unused clocks1662client # [ 0.974284] clk: Disabling unused clocks1663worker1 # [ 0.979248] PM: genpd: Disabling unused power domains1664client # [ 0.974307] PM: genpd: Disabling unused power domains1665client # [ 0.978474] Freeing unused kernel memory: 4736K1666client # [ 0.978690] Run /init as init process1667worker1 # [ 0.983429] Freeing unused kernel memory: 4736K1668worker1 # [ 0.983628] Run /init as init process1669worker1 # [ 0.997921] systemd[1]: Successfully made /usr/ read-only.1670client # [ 0.996534] systemd[1]: Successfully made /usr/ read-only.1671worker2 # [ 1.170871] usb 1-1: new high-speed USB device number 2 using ehci-pci1672lb # [ 1.168457] usb 1-1: new high-speed USB device number 2 using ehci-pci1673client # [ 1.158333] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker1 # [ 1.163728] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker2 # [ 1.323619] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker2 # [ 1.331938] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1677worker2 # [ 1.344322] systemd[1]: Detected virtualization qemu.1678worker2 # [ 1.346403] systemd[1]: Detected architecture arm64.1679worker2 # [ 1.348387] systemd[1]: Running in initrd.1680worker2 # [ 1.351097] systemd[1]: Initializing machine ID from random generator.1681lb # [ 1.321505] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1682worker2 # [ 1.354107] systemd[1]: Hostname set to <worker2>.1683lb # [ 1.334264] systemd[1]: Detected virtualization qemu.1684lb # [ 1.336307] systemd[1]: Detected architecture arm64.1685client # [ 1.310893] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11686worker1 # [ 1.316275] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11687lb # [ 1.339347] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11688lb # [ 1.345355] systemd[1]: Running in initrd.1689lb # [ 1.347944] systemd[1]: Initializing machine ID from random generator.1690lb # [ 1.351173] systemd[1]: Hostname set to <lb>.1691worker1 # [ 1.332807] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692worker1 # [ 1.344921] systemd[1]: Detected virtualization qemu.1693client # [ 1.331370] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1694worker1 # [ 1.346974] systemd[1]: Detected architecture arm64.1695client # [ 1.343566] systemd[1]: Detected virtualization qemu.1696worker1 # [ 1.348978] systemd[1]: Running in initrd.1697client # [ 1.345607] systemd[1]: Detected architecture arm64.1698client # [ 1.347649] systemd[1]: Running in initrd.1699worker1 # [ 1.351595] systemd[1]: Initializing machine ID from random generator.1700client # [ 1.350328] systemd[1]: Initializing machine ID from random generator.1701worker1 # [ 1.354700] systemd[1]: Hostname set to <worker1>.1702client # [ 1.353234] systemd[1]: Hostname set to <client>.1703worker2 # [ 1.403072] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704worker1 # [ 1.399968] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705client # [ 1.398595] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706lb # [ 1.428603] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker2 # [ 1.522830] usb 1-2: new high-speed USB device number 3 using ehci-pci1708worker1 # [ 1.523738] usb 1-2: new high-speed USB device number 3 using ehci-pci1709client # [ 1.522299] usb 1-2: new high-speed USB device number 3 using ehci-pci1710lb # [ 1.548447] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker2 # [ 1.657135] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712lb # [ 1.643165] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713worker2 # [ 1.683171] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21714worker2 # [ 1.685540] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01715worker1 # [ 1.660047] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716client # [ 1.666930] systemd[1]: bpf-restrict-fs: LSM BPF program attached1717lb # [ 1.707136] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718worker2 # [ 1.730944] systemd[1]: Queued start job for default target Initrd Default Target.1719worker1 # [ 1.686319] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21720client # [ 1.683238] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721lb # [ 1.712537] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722worker1 # [ 1.691180] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723client # [ 1.690857] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01724worker2 # [ 1.740974] systemd[1]: Created slice Slice /system/modprobe.1725worker2 # [ 1.742055] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1726worker2 # [ 1.743287] systemd[1]: Expecting device /dev/disk/by-label/nixos...1727worker2 # [ 1.744243] systemd[1]: Reached target Path Units.1728worker2 # [ 1.744987] systemd[1]: Reached target Slice Units.1729worker2 # [ 1.745722] systemd[1]: Reached target Swaps.1730worker2 # [ 1.746485] systemd[1]: Reached target Timer Units.1731worker2 # [ 1.747465] systemd[1]: Listening on D-Bus System Message Bus Socket.1732lb # [ 1.727702] systemd[1]: Queued start job for default target Initrd Default Target.1733worker2 # [ 1.748589] systemd[1]: Listening on Journal Socket (/dev/log).1734worker2 # [ 1.749601] systemd[1]: Listening on Journal Sockets.1735worker2 # [ 1.750482] systemd[1]: Listening on udev Control Socket.1736worker2 # [ 1.751439] systemd[1]: Listening on udev Kernel Socket.1737worker2 # [ 1.751464] systemd[1]: Reached target Socket Units.1738worker2 # [ 1.754453] systemd[1]: Starting Create List of Static Device Nodes...1739worker2 # [ 1.755624] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1740lb # [ 1.738986] systemd[1]: Created slice Slice /system/modprobe.1741lb # [ 1.740272] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1742lb # [ 1.741720] systemd[1]: Expecting device /dev/disk/by-label/nixos...1743lb # [ 1.742786] systemd[1]: Reached target Path Units.1744lb # [ 1.743606] systemd[1]: Reached target Slice Units.1745lb # [ 1.744482] systemd[1]: Reached target Swaps.1746lb # [ 1.745226] systemd[1]: Reached target Timer Units.1747worker2 # [ 1.766027] systemd[1]: Mounting Kernel Configuration File System...1748lb # [ 1.746274] systemd[1]: Listening on D-Bus System Message Bus Socket.1749lb # [ 1.747640] systemd[1]: Listening on Journal Socket (/dev/log).1750lb # [ 1.748859] systemd[1]: Listening on Journal Sockets.1751lb # [ 1.749010] systemd[1]: Listening on udev Control Socket.1752lb # [ 1.749130] systemd[1]: Listening on udev Kernel Socket.1753lb # [ 1.749153] systemd[1]: Reached target Socket Units.1754lb # [ 1.754650] systemd[1]: Starting Create List of Static Device Nodes...1755lb # [ 1.755816] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1756lb # [ 1.764560] systemd[1]: Mounting Kernel Configuration File System...1757worker2 # [ 1.786020] systemd[1]: Starting Journal Service...1758lb # [ 1.772655] systemd[1]: Starting Journal Service...1759worker2 # [ 1.794743] systemd[1]: Starting Load Kernel Modules...1760worker2 # [ 1.795707] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1761worker2 # [ 1.802429] systemd[1]: Starting Coldplug All udev Devices...1762worker1 # [ 1.765843] systemd[1]: Queued start job for default target Initrd Default Target.1763lb # [ 1.795817] systemd[1]: Starting Load Kernel Modules...1764worker1 # [ 1.773986] systemd[1]: Created slice Slice /system/modprobe.1765lb # [ 1.796865] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1766worker1 # [ 1.775165] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1767worker1 # [ 1.776559] systemd[1]: Expecting device /dev/disk/by-label/nixos...1768worker2 # [ 1.817446] systemd[1]: Finished Create List of Static Device Nodes.1769client # [ 1.773327] systemd[1]: Queued start job for default target Initrd Default Target.1770worker1 # [ 1.777586] systemd[1]: Reached target Path Units.1771worker2 # [ 1.818131] systemd[1]: Mounted Kernel Configuration File System.1772worker1 # [ 1.778398] systemd[1]: Reached target Slice Units.1773worker1 # [ 1.779197] systemd[1]: Reached target Swaps.1774worker1 # [ 1.779985] systemd[1]: Reached target Timer Units.1775worker1 # [ 1.780959] systemd[1]: Listening on D-Bus System Message Bus Socket.1776worker1 # [ 1.782135] systemd[1]: Listening on Journal Socket (/dev/log).1777worker1 # [ 1.783232] systemd[1]: Listening on Journal Sockets.1778worker1 # [ 1.784227] systemd[1]: Listening on udev Control Socket.1779worker2 # [ 1.827686] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1780worker1 # [ 1.784360] systemd[1]: Listening on udev Kernel Socket.1781worker1 # [ 1.784383] systemd[1]: Reached target Socket Units.1782client # [ 1.784338] systemd[1]: Created slice Slice /system/modprobe.1783worker1 # [ 1.788526] systemd[1]: Starting Create List of Static Device Nodes...1784client # [ 1.785367] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1785client # [ 1.786547] systemd[1]: Expecting device /dev/disk/by-label/nixos...1786worker1 # [ 1.789645] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1787client # [ 1.787498] systemd[1]: Reached target Path Units.1788client # [ 1.788190] systemd[1]: Reached target Slice Units.1789client # [ 1.788894] systemd[1]: Reached target Swaps.1790client # [ 1.789525] systemd[1]: Reached target Timer Units.1791client # [ 1.790553] systemd[1]: Listening on D-Bus System Message Bus Socket.1792client # [ 1.791600] systemd[1]: Listening on Journal Socket (/dev/log).1793client # [ 1.792556] systemd[1]: Listening on Journal Sockets.1794client # [ 1.793392] systemd[1]: Listening on udev Control Socket.1795worker1 # [ 1.797056] systemd[1]: Mounting Kernel Configuration File System...1796client # [ 1.794288] systemd[1]: Listening on udev Kernel Socket.1797client # [ 1.795063] systemd[1]: Reached target Socket Units.1798client # [ 1.797183] systemd[1]: Starting Create List of Static Device Nodes...1799client # [ 1.798204] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1800lb # [ 1.816422] systemd-journald[72]: Collecting audit messages is disabled.1801lb # [ 1.825720] systemd[1]: Starting Coldplug All udev Devices...1802lb # [ 1.827878] systemd[1]: Finished Create List of Static Device Nodes.1803worker1 # [ 1.807927] systemd[1]: Starting Journal Service...1804client # [ 1.806433] systemd[1]: Mounting Kernel Configuration File System...1805worker2 # [ 1.858007] systemd-journald[72]: Collecting audit messages is disabled.1806client # [ 1.814528] systemd[1]: Starting Journal Service...1807lb # [ 1.841115] systemd[1]: Mounted Kernel Configuration File System.1808worker1 # [ 1.831467] systemd[1]: Starting Load Kernel Modules...1809worker1 # [ 1.832573] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1810worker2 # [ 1.870576] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1811lb # [ 1.852788] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1812client # [ 1.842440] systemd[1]: Starting Load Kernel Modules...1813client # [ 1.843296] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1814client # [ 1.846621] systemd[1]: Starting Coldplug All udev Devices...1815worker2 # [ 1.891389] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1816worker1 # [ 1.847805] systemd[1]: Starting Coldplug All udev Devices...1817worker2 # [ 1.893856] systemd[1]: Starting Create Static Device Nodes in /dev...1818worker2 # [ 1.898104] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1819lb # [ 1.879485] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1820client # [ 1.856294] systemd-journald[72]: Collecting audit messages is disabled.1821client # [ 1.857280] systemd[1]: Finished Create List of Static Device Nodes.1822client # [ 1.858903] systemd[1]: Mounted Kernel Configuration File System.1823worker1 # [ 1.863263] systemd-journald[72]: Collecting audit messages is disabled.1824worker2 # [ 1.908200] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01825client # [ 1.864396] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1826worker2 # [ 1.908434] [drm] features: -virgl +edid -resource_blob -host_visible1827worker2 # [ 1.908444] [drm] features: -context_init1828worker2 # [ 1.909166] [drm] number of scanouts: 11829worker2 # [ 1.909184] [drm] number of cap sets: 01830lb # [ 1.893138] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1831lb # [ 1.894668] systemd[1]: Starting Create Static Device Nodes in /dev...1832worker1 # [ 1.875863] systemd[1]: Finished Create List of Static Device Nodes.1833worker1 # [ 1.876881] systemd[1]: Mounted Kernel Configuration File System.1834lb # [ 1.898145] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1835lb # [ 1.904342] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01836worker1 # [ 1.888095] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1837worker2 # [ 1.935149] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1838lb # [ 1.920635] [drm] features: -virgl +edid -resource_blob -host_visible1839lb # [ 1.920653] [drm] features: -context_init1840worker2 # [ 1.935170] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01841lb # [ 1.921402] [drm] number of scanouts: 11842lb # [ 1.921419] [drm] number of cap sets: 01843worker2 # [ 1.947427] systemd[1]: Finished Create Static Device Nodes in /dev.1844worker2 # [ 1.947626] systemd[1]: Reached target Preparation for Local File Systems.1845worker2 # [ 1.947654] systemd[1]: Reached target Local File Systems.1846worker2 # [ 1.951649] systemd[1]: Starting Rule-based Manager for Device Events and Files...1847client # [ 1.910896] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1848lb # [ 1.932747] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1849worker1 # [ 1.916512] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1850lb # [ 1.932764] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01851client # [ 1.913442] systemd[1]: Starting Create Static Device Nodes in /dev...1852worker2 # [ 1.963070] Console: switching to colour frame buffer device 160x501853lb # [ 1.949183] systemd[1]: Finished Create Static Device Nodes in /dev.1854lb # [ 1.949372] systemd[1]: Reached target Preparation for Local File Systems.1855lb # [ 1.949398] systemd[1]: Reached target Local File Systems.1856worker1 # [ 1.932289] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1857lb # [ 1.953361] systemd[1]: Starting Rule-based Manager for Device Events and Files...1858worker1 # [ 1.934548] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1859worker1 # [ 1.940195] systemd[1]: Starting Create Static Device Nodes in /dev...1860lb # [ 1.964740] Console: switching to colour frame buffer device 160x501861worker1 # [ 1.949808] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01862worker1 # [ 1.950050] [drm] features: -virgl +edid -resource_blob -host_visible1863client # [ 1.947822] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1864worker1 # [ 1.950060] [drm] features: -context_init1865worker1 # [ 1.950886] [drm] number of scanouts: 11866worker1 # [ 1.950905] [drm] number of cap sets: 01867client # [ 1.950808] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1868worker2 # [ 1.995409] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1869lb # [ 1.984965] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1870client # [ 1.963513] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01871client # [ 1.963747] [drm] features: -virgl +edid -resource_blob -host_visible1872worker2 # [ 2.011475] systemd[1]: Finished Load Kernel Modules.1873client # [ 1.963757] [drm] features: -context_init1874client # [ 1.964451] [drm] number of scanouts: 11875worker1 # [ 1.968014] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1876client # [ 1.964468] [drm] number of cap sets: 01877worker1 # [ 1.968032] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01878lb # [ 1.996972] systemd[1]: Finished Load Kernel Modules.1879worker2 # [ 2.019189] systemd[1]: Starting Apply Kernel Variables...1880client # [ 1.979030] systemd[1]: Finished Create Static Device Nodes in /dev.1881client # [ 1.979414] systemd[1]: Reached target Preparation for Local File Systems.1882client # [ 1.979443] systemd[1]: Reached target Local File Systems.1883lb # [ 2.004806] systemd[1]: Starting Apply Kernel Variables...1884client # [ 1.986645] systemd[1]: Starting Rule-based Manager for Device Events and Files...1885lb # [ 2.004512] systemd-modules-load[74]: Inserted module 'dm_mod'1886worker1 # [ 1.992272] Console: switching to colour frame buffer device 160x501887lb # [ 2.005608] systemd-modules-load[74]: Module 'virtio_balloon' is built in1888client # [ 1.990805] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1889lb # [ 2.006692] systemd-modules-load[74]: Module 'virtio_console' is built in1890client # [ 1.990825] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01891lb # [ 2.007736] systemd-modules-load[74]: Inserted module 'virtio_gpu'1892lb # [ 2.025121] systemd[1]: Started Journal Service.1893worker2 # [ 2.036389] systemd-modules-load[73]: Inserted module 'dm_mod'1894worker2 # [ 2.037492] systemd-modules-load[73]: Module 'virtio_balloon' is built in1895worker2 # [ 2.038622] systemd-modules-load[73]: Module 'virtio_console' is built in1896worker2 # [ 2.039669] systemd-modules-load[73]: Inserted module 'virtio_gpu'1897worker1 # [ 2.012327] systemd[1]: Finished Create Static Device Nodes in /dev.1898worker1 # [ 2.012528] systemd[1]: Reached target Preparation for Local File Systems.1899lb # [ 2.020500] systemd-modules-load[74]: Module 'virtio_rng' is built in1900worker1 # [ 2.012554] systemd[1]: Reached target Local File Systems.1901worker2 # [ 2.059039] systemd[1]: Started Journal Service.1902worker1 # [ 2.020105] systemd[1]: Starting Rule-based Manager for Device Events and Files...1903client # [ 2.014546] Console: switching to colour frame buffer device 160x501904lb # [ 2.032121] systemd[1]: Starting Create System Files and Directories...1905worker2 # [ 2.056534] systemd-modules-load[73]: Module 'virtio_rng' is built in1906worker1 # [ 2.028293] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1907worker2 # [ 2.057896] systemd-udevd[78]: Using default interface naming scheme 'v261'.1908worker1 # [ 2.040307] systemd[1]: Finished Load Kernel Modules.1909worker2 # [ 2.068294] systemd[1]: Starting Create System Files and Directories...1910client # [ 2.038477] systemd[1]: Started Journal Service.1911worker2 # [ 2.073242] systemd[1]: Finished Apply Kernel Variables.1912client # [ 2.028491] systemd-modules-load[74]: Inserted module 'dm_mod'1913worker1 # [ 2.045534] systemd[1]: Starting Apply Kernel Variables...1914lb # [ 2.052124] systemd-udevd[79]: Using default interface naming scheme 'v261'.1915lb # [ 2.053857] systemd[1]: Finished Apply Kernel Variables.1916client # [ 2.046946] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1917client # [ 2.035843] systemd-modules-load[74]: Module 'virtio_balloon' is built in1918client # [ 2.041562] systemd-modules-load[74]: Module 'virtio_console' is built in1919client # [ 2.042637] systemd[1]: Starting Create System Files and Directories...1920worker1 # [ 2.045407] systemd-modules-load[74]: Inserted module 'dm_mod'1921client # [ 2.043657] systemd-modules-load[74]: Inserted module 'virtio_gpu'1922worker1 # [ 2.048472] systemd-modules-load[74]: Module 'virtio_balloon' is built in1923worker1 # [ 2.049593] systemd-modules-load[74]: Module 'virtio_console' is built in1924worker1 # [ 2.050675] systemd-modules-load[74]: Inserted module 'virtio_gpu'1925worker1 # [ 2.051682] systemd-modules-load[74]: Module 'virtio_rng' is built in1926lb # [ 2.073642] systemd[1]: Finished Create System Files and Directories.1927worker1 # [ 2.075941] systemd[1]: Started Journal Service.1928worker2 # [ 2.104858] systemd[1]: Started Rule-based Manager for Device Events and Files.1929client # [ 2.060703] systemd-modules-load[74]: Module 'virtio_rng' is built in1930client # [ 2.063966] systemd[1]: Finished Load Kernel Modules.1931worker2 # [ 2.111302] systemd[1]: Finished Create System Files and Directories.1932lb # [ 2.093314] systemd[1]: Started Rule-based Manager for Device Events and Files.1933client # [ 2.079625] systemd[1]: Starting Apply Kernel Variables...1934worker1 # [ 2.088931] systemd[1]: Starting Create System Files and Directories...1935client # [ 2.092502] systemd-udevd[79]: Using default interface naming scheme 'v261'.1936worker1 # [ 2.096750] systemd[1]: Finished Apply Kernel Variables.1937client # [ 2.105535] systemd[1]: Finished Create System Files and Directories.1938worker1 # [ 2.113182] systemd-udevd[78]: Using default interface naming scheme 'v261'.1939worker1 # [ 2.115209] systemd[1]: Finished Create System Files and Directories.1940client # [ 2.121513] systemd[1]: Finished Apply Kernel Variables.1941worker2 # [ 2.174347] systemd[1]: Starting Virtual Console Setup...1942lb # [ 2.152293] systemd[1]: Starting Virtual Console Setup...1943client # [ 2.137259] systemd[1]: Started Rule-based Manager for Device Events and Files.1944worker1 # [ 2.144979] systemd[1]: Started Rule-based Manager for Device Events and Files.1945worker2 # [ 2.224509] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1946lb # [ 2.204542] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1947worker2 # [ 2.227954] systemd[1]: Finished Virtual Console Setup.1948lb # [ 2.208059] systemd[1]: Finished Virtual Console Setup.1949client # [ 2.196788] systemd[1]: Starting Virtual Console Setup...1950worker1 # [ 2.200828] systemd[1]: Starting Virtual Console Setup...1951client # [ 2.252552] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1952worker1 # [ 2.256503] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1953client # [ 2.256007] systemd[1]: Finished Virtual Console Setup.1954worker1 # [ 2.260066] systemd[1]: Finished Virtual Console Setup.1955lb # [ 2.801927] systemd[1]: Finished Coldplug All udev Devices.1956lb # [ 2.804148] systemd[1]: Reached target System Initialization.1957lb # [ 2.808135] systemd[1]: Reached target Basic System.1958worker2 # [ 2.844795] systemd[1]: Finished Coldplug All udev Devices.1959worker2 # [ 2.845781] systemd[1]: Reached target System Initialization.1960worker2 # [ 2.848164] systemd[1]: Reached target Basic System.1961client # [ 2.860371] systemd[1]: Finished Coldplug All udev Devices.1962client # [ 2.861475] systemd[1]: Reached target System Initialization.1963client # [ 2.862321] systemd[1]: Reached target Basic System.1964worker1 # [ 2.879545] systemd[1]: Finished Coldplug All udev Devices.1965worker1 # [ 2.880594] systemd[1]: Reached target System Initialization.1966worker1 # [ 2.881402] systemd[1]: Reached target Basic System.1967lb # [ 2.971545] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1968worker2 # [ 2.995780] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1969lb # [ 2.976500] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1970lb # [ 2.988704] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1971worker2 # [ 3.025399] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972worker2 # [ 3.028643] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1973client # [ 3.036273] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1974client # [ 3.041420] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975lb # [ 3.066049] systemd[1]: Found device /dev/disk/by-label/nixos.1976worker1 # [ 3.048573] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1977client # [ 3.048191] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1978lb # [ 3.072089] systemd[1]: Reached target Initrd Root Device.1979worker1 # [ 3.052726] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1980lb # [ 3.076890] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1981worker1 # [ 3.058719] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1982worker2 # [ 3.120380] systemd[1]: Found device /dev/disk/by-label/nixos.1983worker2 # [ 3.121780] systemd[1]: Reached target Initrd Root Device.1984worker2 # [ 3.124982] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985lb # [ 3.123922] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1986lb # [ 3.131625] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1987lb # [ 3.134439] systemd[1]: Mounting /sysroot...1988client # [ 3.120064] systemd[1]: Found device /dev/disk/by-label/nixos.1989client # [ 3.125081] systemd[1]: Reached target Initrd Root Device.1990worker2 # [ 3.170050] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1991client # [ 3.130976] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992worker2 # [ 3.178680] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993worker1 # [ 3.136125] systemd[1]: Found device /dev/disk/by-label/nixos.1994worker1 # [ 3.140175] systemd[1]: Reached target Initrd Root Device.1995worker2 # [ 3.183898] systemd[1]: Mounting /sysroot...1996worker1 # [ 3.145770] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1997lb # [ 3.191184] EXT4-fs (vda): mounted filesystem ad359d95-b0e5-48b9-a22f-93ffc0000a4d r/w with ordered data mode. Quota mode: none.1998lb # [ 3.180040] systemd[1]: Mounted /sysroot.1999lb # [ 3.182054] systemd[1]: Reached target Initrd Root File System.2000lb # [ 3.185549] systemd[1]: Starting Mountpoints Configured in the Real Root...
loading 2,094 hidden lines…
4095worker1 # [ 39.806467] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4096worker1 # [ 39.828912] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5)4097worker1 # [ 39.926200] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 2/5)4098worker1 # [ 40.102155] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 240 ms (attempt 3/5)4099worker1 # [ 40.344356] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 574 ms (attempt 4/5)4100worker1 # [ 40.922169] nix-grpc-daemon[768]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4101worker1 # [ 40.928257] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4102worker2 # [ 41.008253] nix-grpc-daemon[769]: ts=2026-09-22T10:59:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:362024103worker2 # [ 41.020915] systemd[1]: Started Nix Daemon.4104worker2 # [ 41.239156] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4105worker2 # [ 41.272397] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)4106worker2 # [ 41.356694] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 2/5)4107worker2 # [ 41.446652] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 121 ms (attempt 3/5)4108worker2 # [ 41.569664] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 616 ms (attempt 4/5)4109worker2 # [ 42.189336] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4110worker2 # [ 42.195020] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4111worker2 # [ 42.207296] nix-grpc-daemon[769]: ts=2026-09-22T10:59:07Z level=debug event=path_info_miss path=ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test error="error: path '/nix/store/ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test' is required, but there is no substituter that can build it"4112worker2 # [ 42.222352] nix-grpc-daemon[769]: ts=2026-09-22T10:59:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:362024113worker2 # [ 42.230350] nix-grpc-daemon[769]: ts=2026-09-22T10:59:07Z level=debug event=path_info_miss path=ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test error="error: path '/nix/store/ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test' is required, but there is no substituter that can build it"4114client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4115client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4116client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4117client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4118worker1 # [ 42.519965] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=info event=oidc_jwks provider=mock keys=14119worker1 # [ 42.521907] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:479944120worker1 # [ 42.541026] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:479944121worker1 # [ 42.555315] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=path_info_miss path=ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test error="error: path '/nix/store/ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test' is required, but there is no substituter that can build it"4122worker1 # [ 42.571762] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:479944123worker1 # [ 42.583667] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=path_info_miss path=ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test error="error: path '/nix/store/ylfbn6z3ahrh8hrrlqp2284fnn7jlrh4-nixos-system-client-test' is required, but there is no substituter that can build it"4124client: must succeed: rm -r /var/lib/nix-grpc-store4125client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4126worker2 # [ 42.873430] nix-grpc-daemon[769]: ts=2026-09-22T10:59:08Z level=info event=denied method=QueryMissing cn=- role=none4127worker2 # [ 43.041975] nix-grpc-daemon[769]: ts=2026-09-22T10:59:08Z level=info event=denied method=QueryMissing cn=stranger role=none4128(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.70 seconds)4129subtest: DAG build is scheduled across workers and lands in the cache4130client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24131worker1 # [ 43.168469] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479944132client # don't know how to build these paths:4133client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4134worker1 # [ 43.178558] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944135worker2 # [ 43.263466] nix-grpc-daemon[769]: ts=2026-09-22T10:59:08Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04136worker2 # [ 43.270359] nix-grpc-daemon[769]: ts=2026-09-22T10:59:08Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=14137worker1 # [ 43.267609] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944138worker1 # [ 43.289622] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944139client # copying 1 paths to 'lb:50051'...4140worker1 # [ 43.305658] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479944141client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4142worker1 # [ 43.332905] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4143worker1 # [ 43.371854] nix-grpc-daemon[768]: ts=2026-09-22T10:59:08Z level=info event=niks3_push_started pid=8474144worker1 # [ 43.559332] nix-grpc-daemon[847]: time=2026-09-22T10:59:09.153Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4145worker1 # [ 43.672633] nix-daemon[834]: accepted connection from pid 860, user nix-grpc-daemon (trusted)4146worker1 # [ 43.695358] nix-daemon[834]: reaped child process 863, status = succeeded4147worker1 # [ 43.794637] nix-daemon[834]: accepted connection from pid 865, user nix-grpc-daemon (trusted)4148worker1 # [ 43.814381] nix-daemon[834]: reaped child process 868, status = succeeded4149lb # [ 43.834282] niks3-server[998]: 2026/09/22 10:59:09 INFO Received uploads request method=POST path=/api/pending_closures4150worker1 # [ 43.833016] nix-grpc-daemon[847]: time=2026-09-22T10:59:09.426Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4151worker1 # [ 43.834585] nix-grpc-daemon[847]: time=2026-09-22T10:59:09.426Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4152lb # [ 43.900916] niks3-server[998]: 2026/09/22 10:59:09 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4153lb # [ 43.908398] niks3-server[998]: 2026/09/22 10:59:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4154lb # [ 43.928220] niks3-server[998]: 2026/09/22 10:59:09 INFO Signed narinfos id=1 count=14155worker1 # [ 43.911036] nix-grpc-daemon[847]: time=2026-09-22T10:59:09.505Z level=INFO msg="Uploading 1 narinfos"4156lb # [ 43.946866] niks3-server[998]: 2026/09/22 10:59:09 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4157lb # [ 43.977016] niks3-server[998]: 2026/09/22 10:59:09 INFO Completed upload id=14158worker1 # [ 43.955279] nix-grpc-daemon[847]: time=2026-09-22T10:59:09.550Z level=INFO msg="Upload complete. (391ms)"4159lb # [ 43.985363] niks3-server[998]: 2026/09/22 10:59:09 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4160worker1 # [ 43.969259] nix-grpc-daemon[768]: ts=2026-09-22T10:59:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_in=5044161worker1 # [ 43.987806] nix-grpc-daemon[768]: ts=2026-09-22T10:59:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944162worker1 # [ 43.999458] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4163worker1 # [ 44.003279] nix-daemon[834]: reaped child process 845, status = succeeded4164client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4165client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4166worker1 # [ 44.020650] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4167client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4168client # farm-a> LOG-farm-a-t14169worker1 # [ 45.319508] nix-daemon[834]: reaped child process 872, status = succeeded4170worker1 # [ 45.387547] nix-daemon[834]: accepted connection from pid 889, user nix-grpc-daemon (trusted)4171worker1 # [ 45.408290] nix-daemon[834]: reaped child process 892, status = succeeded4172lb # [ 45.433438] niks3-server[998]: 2026/09/22 10:59:11 INFO Received uploads request method=POST path=/api/pending_closures4173worker1 # [ 45.428496] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.021Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4174worker1 # [ 45.430229] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.021Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4175lb # [ 45.500866] niks3-server[998]: 2026/09/22 10:59:11 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4176lb # [ 45.508881] niks3-server[998]: 2026/09/22 10:59:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4177worker1 # [ 45.494562] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.088Z level=INFO msg="Uploading 1 narinfos"4178lb # [ 45.517266] niks3-server[998]: 2026/09/22 10:59:11 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4179lb # [ 45.524868] niks3-server[998]: 2026/09/22 10:59:11 INFO Signed narinfos id=2 count=14180lb # [ 45.526728] niks3-server[998]: 2026/09/22 10:59:11 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4181lb # [ 45.538484] niks3-server[998]: 2026/09/22 10:59:11 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4182lb # [ 45.544825] niks3-server[998]: 2026/09/22 10:59:11 INFO Completed upload id=24183worker1 # [ 45.523322] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.118Z level=INFO msg="Upload complete. (290ms)"4184lb # [ 45.547836] niks3-server[998]: 2026/09/22 10:59:11 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4185worker1 # [ 45.537176] nix-grpc-daemon[768]: ts=2026-09-22T10:59:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14186worker2 # [ 45.588674] nix-grpc-daemon[769]: ts=2026-09-22T10:59:11Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04187worker2 # [ 45.594164] nix-grpc-daemon[769]: ts=2026-09-22T10:59:11Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14188worker1 # [ 45.556487] nix-daemon[834]: reaped child process 870, status = succeeded4189worker1 # [ 45.591778] nix-grpc-daemon[768]: ts=2026-09-22T10:59:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944190worker1 # [ 45.607457] nix-grpc-daemon[768]: ts=2026-09-22T10:59:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944191worker1 # [ 45.623075] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.218Z level=INFO msg="All 1 paths already cached"4192client # copying 1 paths to 'lb:50051'...4193worker1 # [ 45.633516] nix-grpc-daemon[768]: ts=2026-09-22T10:59:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479944194client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4195worker1 # [ 45.655379] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4196worker1 # [ 45.825439] nix-daemon[834]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4197worker1 # [ 45.849436] nix-daemon[834]: reaped child process 899, status = succeeded4198worker1 # [ 45.948662] nix-daemon[834]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4199worker1 # [ 45.967183] nix-daemon[834]: reaped child process 904, status = succeeded4200lb # [ 45.992341] niks3-server[998]: 2026/09/22 10:59:11 INFO Received uploads request method=POST path=/api/pending_closures4201worker1 # [ 45.980799] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.575Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4202worker1 # [ 45.982312] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.575Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4203lb # [ 46.020188] niks3-server[998]: 2026/09/22 10:59:11 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4204worker1 # [ 46.011806] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.606Z level=INFO msg="Uploading 1 narinfos"4205lb # [ 46.034760] niks3-server[998]: 2026/09/22 10:59:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4206lb # [ 46.046724] niks3-server[998]: 2026/09/22 10:59:11 INFO Signed narinfos id=3 count=14207lb # [ 46.053364] niks3-server[998]: 2026/09/22 10:59:11 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4208lb # [ 46.065074] niks3-server[998]: 2026/09/22 10:59:11 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4209lb # [ 46.073952] niks3-server[998]: 2026/09/22 10:59:11 INFO Completed upload id=34210worker1 # [ 46.051732] nix-grpc-daemon[847]: time=2026-09-22T10:59:11.646Z level=INFO msg="Upload complete. (378ms)"4211lb # [ 46.078202] niks3-server[998]: 2026/09/22 10:59:11 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4212worker1 # [ 46.064285] nix-grpc-daemon[768]: ts=2026-09-22T10:59:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_in=6244213worker1 # [ 46.076255] nix-grpc-daemon[768]: ts=2026-09-22T10:59:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944214worker1 # [ 46.088320] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4215worker1 # [ 46.091555] nix-daemon[834]: reaped child process 894, status = succeeded4216client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4217client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4218worker1 # [ 46.106772] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4219client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4220client # farm-b> LOG-farm-b-t14221worker1 # [ 47.412511] nix-daemon[834]: reaped child process 908, status = succeeded4222worker1 # [ 47.479125] nix-daemon[834]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4223worker1 # [ 47.501172] nix-daemon[834]: reaped child process 927, status = succeeded4224lb # [ 47.522147] niks3-server[998]: 2026/09/22 10:59:13 INFO Received uploads request method=POST path=/api/pending_closures4225worker1 # [ 47.511596] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.106Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4226worker1 # [ 47.514233] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.108Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4227lb # [ 47.567066] niks3-server[998]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4228worker1 # [ 47.560428] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.155Z level=INFO msg="Uploading 1 narinfos"4229lb # [ 47.582367] niks3-server[998]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4230lb # [ 47.597469] niks3-server[998]: 2026/09/22 10:59:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4231lb # [ 47.617510] niks3-server[998]: 2026/09/22 10:59:13 INFO Signed narinfos id=4 count=14232worker1 # [ 47.601984] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.197Z level=INFO msg="Upload complete. (283ms)"4233lb # [ 47.629267] niks3-server[998]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4234lb # [ 47.633675] niks3-server[998]: 2026/09/22 10:59:13 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4235lb # [ 47.637352] niks3-server[998]: 2026/09/22 10:59:13 INFO Completed upload id=44236worker1 # [ 47.613219] nix-grpc-daemon[768]: ts=2026-09-22T10:59:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14237lb # [ 47.641964] niks3-server[998]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4238worker2 # [ 47.665045] nix-grpc-daemon[769]: ts=2026-09-22T10:59:13Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04239worker2 # [ 47.671081] nix-grpc-daemon[769]: ts=2026-09-22T10:59:13Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14240worker1 # [ 47.634440] nix-daemon[834]: reaped child process 906, status = succeeded4241worker1 # [ 47.671182] nix-grpc-daemon[768]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944242worker1 # [ 47.686933] nix-grpc-daemon[768]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944243worker1 # [ 47.700917] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.296Z level=INFO msg="All 2 paths already cached"4244client # copying 1 paths to 'lb:50051'...4245client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4246worker1 # [ 47.711070] nix-grpc-daemon[768]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479944247worker1 # [ 47.729282] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4248worker1 # [ 47.890268] nix-daemon[834]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4249worker1 # [ 47.912742] nix-daemon[834]: reaped child process 934, status = succeeded4250worker1 # [ 48.012566] nix-daemon[834]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4251worker1 # [ 48.032483] nix-daemon[834]: reaped child process 939, status = succeeded4252lb # [ 48.055297] niks3-server[998]: 2026/09/22 10:59:13 INFO Received uploads request method=POST path=/api/pending_closures4253worker1 # [ 48.043794] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.638Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4254worker1 # [ 48.045513] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.638Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4255worker1 # [ 48.071312] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.666Z level=INFO msg="Uploading 1 narinfos"4256lb # [ 48.092175] niks3-server[998]: 2026/09/22 10:59:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4257lb # [ 48.103903] niks3-server[998]: 2026/09/22 10:59:13 INFO Signed narinfos id=5 count=14258lb # [ 48.114466] niks3-server[998]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4259lb # [ 48.127790] niks3-server[998]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4260worker1 # [ 48.105900] nix-grpc-daemon[847]: time=2026-09-22T10:59:13.701Z level=INFO msg="Upload complete. (358ms)"4261worker1 # [ 48.112449] nix-grpc-daemon[768]: ts=2026-09-22T10:59:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_in=7524262lb # [ 48.138726] niks3-server[998]: 2026/09/22 10:59:13 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4263worker1 # [ 48.120779] nix-grpc-daemon[768]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944264lb # [ 48.144506] niks3-server[998]: 2026/09/22 10:59:13 INFO Completed upload id=54265lb # [ 48.146963] niks3-server[998]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4266worker1 # [ 48.132669] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4267worker1 # [ 48.135351] nix-daemon[834]: reaped child process 929, status = succeeded4268client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4269worker1 # [ 48.147066] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4270client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4271lb # [ 48.789388] envoy[898]: conn peer=[2001:db8:1::1]:56802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=26224272lb # [ 48.794548] envoy[898]: conn peer=[2001:db8:1::1]:58602 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=914273lb # [ 48.799223] envoy[898]: conn peer=[2001:db8:1::1]:58608 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114274lb # [ 48.803883] envoy[898]: conn peer=[2001:db8:1::1]:58612 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134275client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4276client # farm-top> LOG-farm-top-t14277worker1 # [ 49.655272] nix-daemon[834]: reaped child process 943, status = succeeded4278worker1 # [ 49.727029] nix-daemon[834]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4279worker1 # [ 49.746785] nix-daemon[834]: reaped child process 962, status = succeeded4280lb # [ 49.773431] niks3-server[998]: 2026/09/22 10:59:15 INFO Received uploads request method=POST path=/api/pending_closures4281worker1 # [ 49.764205] nix-grpc-daemon[847]: time=2026-09-22T10:59:15.357Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4282worker1 # [ 49.765909] nix-grpc-daemon[847]: time=2026-09-22T10:59:15.357Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4283lb # [ 49.818926] niks3-server[998]: 2026/09/22 10:59:15 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4284lb # [ 49.829280] niks3-server[998]: 2026/09/22 10:59:15 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4285worker1 # [ 49.818026] nix-grpc-daemon[847]: time=2026-09-22T10:59:15.412Z level=INFO msg="Uploading 1 narinfos"4286lb # [ 49.842291] niks3-server[998]: 2026/09/22 10:59:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4287lb # [ 49.849827] niks3-server[998]: 2026/09/22 10:59:15 INFO Signed narinfos id=6 count=14288lb # [ 49.854710] niks3-server[998]: 2026/09/22 10:59:15 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4289lb # [ 49.864632] niks3-server[998]: 2026/09/22 10:59:15 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4290worker1 # [ 49.850066] nix-grpc-daemon[847]: time=2026-09-22T10:59:15.445Z level=INFO msg="Upload complete. (293ms)"4291lb # [ 49.874233] niks3-server[998]: 2026/09/22 10:59:15 INFO Completed upload id=64292lb # [ 49.878882] niks3-server[998]: 2026/09/22 10:59:15 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4293worker1 # [ 49.861175] nix-grpc-daemon[768]: ts=2026-09-22T10:59:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14294worker1 # [ 49.881190] nix-daemon[834]: reaped child process 941, status = succeeded4295client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.89 seconds)4296client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4297client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds)4298client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4299client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4300client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4301client # copying 3 paths...4302client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4303client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4304client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4305client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)4306(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.44 seconds)4307subtest: repeat is answered Cached by the scheduler without building4308worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4309worker1 # finding garbage collector roots...4310worker1 # deleting specified paths...4311worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4312worker1 # deleting unused links...4313worker1 # note: hard linking is currently saving 0.0 KiB4314worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4315worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4316worker2 # finding garbage collector roots...4317worker2 # deleting specified paths...4318worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4319worker2 # deleting unused links...4320worker2 # note: hard linking is currently saving 0.0 KiB4321worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)4322client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24323worker1 # [ 51.138191] nix-grpc-daemon[768]: ts=2026-09-22T10:59:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479944324client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4325client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4326worker1 # [ 51.155188] nix-grpc-daemon[768]: ts=2026-09-22T10:59:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944327worker1 # [ 51.169035] nix-grpc-daemon[847]: time=2026-09-22T10:59:16.762Z level=INFO msg="All 2 paths already cached"4328worker2 # [ 51.226845] nix-grpc-daemon[769]: ts=2026-09-22T10:59:16Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14329client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.31 seconds)4330client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4331client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.15 seconds)4332lb: must succeed: curl -sf localhost:9901/clusters4333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4334lb: must succeed: curl -sf localhost:9901/clusters4335lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4336worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4337worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4338(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.19 seconds)4339subtest: inputs already in the cache are not sent to the scheduler4340lb: must succeed: curl -sf localhost:9901/clusters4341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4342lb: must succeed: curl -sf localhost:9901/clusters4343lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4344worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4345worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4346client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24347worker1 # [ 52.023323] nix-grpc-daemon[768]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479944348client # don't know how to build these paths:4349client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4350worker1 # [ 52.032864] nix-grpc-daemon[768]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944351worker1 # [ 52.043409] nix-grpc-daemon[847]: time=2026-09-22T10:59:17.638Z level=INFO msg="All 2 paths already cached"4352worker2 # [ 52.101906] nix-grpc-daemon[769]: ts=2026-09-22T10:59:17Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04353worker2 # [ 52.107934] nix-grpc-daemon[769]: ts=2026-09-22T10:59:17Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14354worker1 # [ 52.107439] nix-grpc-daemon[768]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944355worker1 # [ 52.123011] nix-grpc-daemon[768]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944356worker1 # [ 52.138239] nix-grpc-daemon[847]: time=2026-09-22T10:59:17.733Z level=INFO msg="All 2 paths already cached"4357client # copying 1 paths to 'lb:50051'...4358client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4359worker1 # [ 52.149861] nix-grpc-daemon[768]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479944360worker1 # [ 52.166932] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4361worker1 # [ 52.324557] nix-daemon[834]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4362worker1 # [ 52.344925] nix-daemon[834]: reaped child process 982, status = succeeded4363worker1 # [ 52.452263] nix-daemon[834]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4364worker1 # [ 52.472204] nix-daemon[834]: reaped child process 987, status = succeeded4365lb # [ 52.497619] niks3-server[998]: 2026/09/22 10:59:18 INFO Received uploads request method=POST path=/api/pending_closures4366worker1 # [ 52.488814] nix-grpc-daemon[847]: time=2026-09-22T10:59:18.083Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4367worker1 # [ 52.490320] nix-grpc-daemon[847]: time=2026-09-22T10:59:18.083Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4368lb # [ 52.535856] niks3-server[998]: 2026/09/22 10:59:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4369worker1 # [ 52.525119] nix-grpc-daemon[847]: time=2026-09-22T10:59:18.120Z level=INFO msg="Uploading 1 narinfos"4370lb # [ 52.549573] niks3-server[998]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4371lb # [ 52.560945] niks3-server[998]: 2026/09/22 10:59:18 INFO Signed narinfos id=7 count=14372lb # [ 52.572137] niks3-server[998]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4373lb # [ 52.575959] niks3-server[998]: 2026/09/22 10:59:18 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4374lb # [ 52.584367] niks3-server[998]: 2026/09/22 10:59:18 INFO Completed upload id=74375lb # [ 52.588143] niks3-server[998]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4376worker1 # [ 52.562362] nix-grpc-daemon[847]: time=2026-09-22T10:59:18.157Z level=INFO msg="Upload complete. (379ms)"4377worker1 # [ 52.573903] nix-grpc-daemon[768]: ts=2026-09-22T10:59:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_in=7524378worker1 # [ 52.583659] nix-grpc-daemon[768]: ts=2026-09-22T10:59:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944379worker1 # [ 52.593614] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4380worker1 # [ 52.596751] nix-daemon[834]: reaped child process 977, status = succeeded4381client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4382worker1 # [ 52.610908] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4383client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4384client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4385client # farm-top> LOG-farm-top-t1b4386worker1 # [ 53.242492] nix-daemon[834]: reaped child process 991, status = succeeded4387worker1 # [ 53.341399] nix-daemon[834]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)4388worker1 # [ 53.360418] nix-daemon[834]: reaped child process 1010, status = succeeded4389lb # [ 53.385454] niks3-server[998]: 2026/09/22 10:59:18 INFO Received uploads request method=POST path=/api/pending_closures4390worker1 # [ 53.376079] nix-grpc-daemon[847]: time=2026-09-22T10:59:18.969Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4391worker1 # [ 53.377555] nix-grpc-daemon[847]: time=2026-09-22T10:59:18.969Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4392lb # [ 53.423202] niks3-server[998]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4393lb # [ 53.440117] niks3-server[998]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4394lb # [ 53.453754] niks3-server[998]: 2026/09/22 10:59:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4395worker1 # [ 53.439520] nix-grpc-daemon[847]: time=2026-09-22T10:59:19.034Z level=INFO msg="Uploading 1 narinfos"4396lb # [ 53.466178] niks3-server[998]: 2026/09/22 10:59:19 INFO Signed narinfos id=8 count=14397lb # [ 53.472215] niks3-server[998]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4398worker1 # [ 53.464277] nix-grpc-daemon[847]: time=2026-09-22T10:59:19.059Z level=INFO msg="Upload complete. (244ms)"4399lb # [ 53.484585] niks3-server[998]: 2026/09/22 10:59:19 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4400lb # [ 53.495938] niks3-server[998]: 2026/09/22 10:59:19 INFO Completed upload id=84401worker1 # [ 53.477084] nix-grpc-daemon[768]: ts=2026-09-22T10:59:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14402lb # [ 53.502288] niks3-server[998]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4403worker1 # [ 53.494767] nix-daemon[834]: reaped child process 989, status = succeeded4404client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.71 seconds)4405client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4406client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds)4407lb: must succeed: curl -sf localhost:9901/clusters4408lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4409lb: must succeed: curl -sf localhost:9901/clusters4410lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4411worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4412worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4413(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.22 seconds)4414subtest: two clients wanting the same drv build it once4415client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4416client # [ 53.930169] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4417client # Running as unit: dedup1.service; invocation ID: 858319fc617945e5a4d7e6d53cb3bcc44418client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.09 seconds)4419client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4420client # [ 54.133154] nix[927]: don't know how to build these paths:4421client # [ 54.134250] nix[927]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4422worker1 # [ 54.133822] nix-grpc-daemon[768]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479944423worker2 # [ 54.191294] nix-grpc-daemon[769]: ts=2026-09-22T10:59:19Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04424worker2 # [ 54.193310] nix-grpc-daemon[769]: ts=2026-09-22T10:59:19Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14425worker2 # [ 54.193500] nix-grpc-daemon[769]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024426worker2 # [ 54.220552] nix-grpc-daemon[769]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024427client # [ 54.187101] nix[927]: copying 1 paths to 'lb:50051'...4428client # [ 54.190569] nix[927]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4429worker2 # [ 54.235619] nix-grpc-daemon[769]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024430worker2 # [ 54.250932] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4431worker2 # [ 54.278091] nix-grpc-daemon[769]: ts=2026-09-22T10:59:19Z level=info event=niks3_push_started pid=9074432worker2 # [ 54.445993] nix-grpc-daemon[907]: time=2026-09-22T10:59:20.044Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4433worker2 # [ 54.559318] nix-daemon[855]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4434worker2 # [ 54.579675] nix-daemon[855]: reaped child process 923, status = succeeded4435worker2 # [ 54.686909] nix-daemon[855]: accepted connection from pid 925, user nix-grpc-daemon (trusted)4436worker2 # [ 54.704823] nix-daemon[855]: reaped child process 928, status = succeeded4437lb # [ 54.689040] niks3-server[998]: 2026/09/22 10:59:20 INFO Received uploads request method=POST path=/api/pending_closures4438worker2 # [ 54.721904] nix-grpc-daemon[907]: time=2026-09-22T10:59:20.320Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4439worker2 # [ 54.723394] nix-grpc-daemon[907]: time=2026-09-22T10:59:20.320Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4440lb # [ 54.729387] niks3-server[998]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4441worker2 # [ 54.763465] nix-grpc-daemon[907]: time=2026-09-22T10:59:20.362Z level=INFO msg="Uploading 1 narinfos"4442lb # [ 54.744764] niks3-server[998]: 2026/09/22 10:59:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4443lb # [ 54.754113] niks3-server[998]: 2026/09/22 10:59:20 INFO Signed narinfos id=9 count=14444lb # [ 54.759597] niks3-server[998]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4445lb # [ 54.769397] niks3-server[998]: 2026/09/22 10:59:20 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4446worker2 # [ 54.797042] nix-grpc-daemon[907]: time=2026-09-22T10:59:20.396Z level=INFO msg="Upload complete. (346ms)"4447lb # [ 54.777482] niks3-server[998]: 2026/09/22 10:59:20 INFO Completed upload id=94448lb # [ 54.781753] niks3-server[998]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4449worker2 # [ 54.808120] nix-grpc-daemon[769]: ts=2026-09-22T10:59:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=4324450worker2 # [ 54.819042] nix-grpc-daemon[769]: ts=2026-09-22T10:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024451worker2 # [ 54.826548] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4452worker2 # [ 54.831888] nix-daemon[855]: reaped child process 905, status = succeeded4453client # [ 54.798269] nix[927]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4454worker2 # [ 54.845982] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4455client # [ 55.041651] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4456client # Running as unit: dedup2.service; invocation ID: 18fce0d980bd42a192a7dc89cbb0bb074457client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.12 seconds)4458worker1 # [ 55.278377] nix-grpc-daemon[768]: ts=2026-09-22T10:59:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479944459client # [ 55.286689] nix[943]: don't know how to build these paths:4460client # [ 55.287609] nix[943]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4461worker2 # [ 55.341700] nix-grpc-daemon[769]: ts=2026-09-22T10:59:20Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04462client # [ 55.303274] nix[943]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4463worker2 # [ 55.348315] nix-grpc-daemon[769]: ts=2026-09-22T10:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024464client: must succeed: sleep 24465client: (finished: must succeed: sleep 2, in 2.05 seconds)4466client: waiting for success: ! systemctl is-active dedup1 dedup24467worker2 # [ 58.679406] nix-daemon[855]: reaped child process 932, status = succeeded4468worker2 # [ 58.776882] nix-daemon[855]: accepted connection from pid 964, user nix-grpc-daemon (trusted)4469worker2 # [ 58.794162] nix-daemon[855]: reaped child process 967, status = succeeded4470lb # [ 58.779617] niks3-server[998]: 2026/09/22 10:59:24 INFO Received uploads request method=POST path=/api/pending_closures4471lb # [ 58.789858] envoy[898]: conn peer=[2001:db8:1::1]:58624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9605 tx=4111 ms=67154472worker2 # [ 58.818091] nix-grpc-daemon[907]: time=2026-09-22T10:59:24.417Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4473worker2 # [ 58.822837] nix-grpc-daemon[907]: time=2026-09-22T10:59:24.419Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4474lb # [ 58.796605] envoy[898]: conn peer=[2001:db8:1::1]:47662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=704475lb # [ 58.806619] envoy[898]: conn peer=[2001:db8:1::1]:47670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1783 ms=14654476lb # [ 58.827538] niks3-server[998]: 2026/09/22 10:59:24 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4477lb # [ 58.842260] niks3-server[998]: 2026/09/22 10:59:24 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4478lb # [ 58.856185] niks3-server[998]: 2026/09/22 10:59:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4479worker2 # [ 58.884100] nix-grpc-daemon[907]: time=2026-09-22T10:59:24.482Z level=INFO msg="Uploading 1 narinfos"4480lb # [ 58.863536] niks3-server[998]: 2026/09/22 10:59:24 INFO Signed narinfos id=10 count=14481lb # [ 58.870800] niks3-server[998]: 2026/09/22 10:59:24 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4482lb # [ 58.882202] niks3-server[998]: 2026/09/22 10:59:24 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4483lb # [ 58.889764] niks3-server[998]: 2026/09/22 10:59:24 INFO Completed upload id=104484worker2 # [ 58.911350] nix-grpc-daemon[907]: time=2026-09-22T10:59:24.510Z level=INFO msg="Upload complete. (253ms)"4485lb # [ 58.893818] niks3-server[998]: 2026/09/22 10:59:24 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4486worker2 # [ 58.925190] nix-grpc-daemon[769]: ts=2026-09-22T10:59:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14487worker2 # [ 58.944633] nix-grpc-daemon[769]: ts=2026-09-22T10:59:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14488worker2 # [ 58.952315] nix-daemon[855]: reaped child process 930, status = succeeded4489client # [ 58.913911] systemd[1]: dedup1.service: Deactivated successfully.4490client # [ 58.919624] systemd[1]: dedup1.service: Consumed 174ms CPU time over 4.985s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.4491client # [ 58.968817] systemd[1]: dedup2.service: Deactivated successfully.4492client # [ 58.972389] systemd[1]: dedup2.service: Consumed 212ms CPU time over 3.932s wall clock time, 35.3M memory peak, 5.8K incoming IP traffic, 5.6K outgoing IP traffic.4493client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4494client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4495client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4496worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4497worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4498worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4499worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4500worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4501worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4502worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4503worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4504(finished: subtest: two clients wanting the same drv build it once, in 6.16 seconds)4505subtest: --timeout and --max-silent-time reach the builder4506worker2 # [ 60.281041] nix-grpc-daemon[769]: ts=2026-09-22T10:59:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362024507worker2 # [ 60.297985] nix-grpc-daemon[769]: ts=2026-09-22T10:59:25Z level=debug event=want client=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04508worker2 # [ 60.303844] nix-grpc-daemon[769]: ts=2026-09-22T10:59:25Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14509worker1 # [ 60.305022] nix-grpc-daemon[768]: ts=2026-09-22T10:59:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944510worker1 # [ 60.321016] nix-grpc-daemon[768]: ts=2026-09-22T10:59:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944511worker1 # [ 60.335822] nix-grpc-daemon[768]: ts=2026-09-22T10:59:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479944512worker1 # [ 60.350982] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4513worker1 # [ 60.516660] nix-daemon[834]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4514worker1 # [ 60.543098] nix-daemon[834]: reaped child process 1038, status = succeeded4515worker1 # [ 60.639697] nix-daemon[834]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)4516worker1 # [ 60.657360] nix-daemon[834]: reaped child process 1043, status = succeeded4517lb # [ 60.684664] niks3-server[998]: 2026/09/22 10:59:26 INFO Received uploads request method=POST path=/api/pending_closures4518worker1 # [ 60.672876] nix-grpc-daemon[847]: time=2026-09-22T10:59:26.267Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4519worker1 # [ 60.674397] nix-grpc-daemon[847]: time=2026-09-22T10:59:26.267Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4520lb # [ 60.717510] niks3-server[998]: 2026/09/22 10:59:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4521worker1 # [ 60.704591] nix-grpc-daemon[847]: time=2026-09-22T10:59:26.299Z level=INFO msg="Uploading 1 narinfos"4522lb # [ 60.731429] niks3-server[998]: 2026/09/22 10:59:26 INFO Signed narinfos id=11 count=14523lb # [ 60.738448] niks3-server[998]: 2026/09/22 10:59:26 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4524lb # [ 60.751663] niks3-server[998]: 2026/09/22 10:59:26 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4525worker1 # [ 60.738029] nix-grpc-daemon[847]: time=2026-09-22T10:59:26.333Z level=INFO msg="Upload complete. (370ms)"4526lb # [ 60.763806] niks3-server[998]: 2026/09/22 10:59:26 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4527worker1 # [ 60.745438] nix-grpc-daemon[768]: ts=2026-09-22T10:59:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_in=4244528lb # [ 60.769926] niks3-server[998]: 2026/09/22 10:59:26 INFO Completed upload id=114529lb # [ 60.773615] niks3-server[998]: 2026/09/22 10:59:26 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4530worker1 # [ 60.760367] nix-grpc-daemon[768]: ts=2026-09-22T10:59:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944531worker1 # [ 60.767791] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4532worker1 # [ 60.773525] nix-daemon[834]: reaped child process 1033, status = succeeded4533worker1 # [ 60.787607] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4534worker1 # [ 66.802963] nix-grpc-daemon[768]: ts=2026-09-22T10:59:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=6 outputs=04535worker1 # [ 66.821906] nix-daemon[834]: reaped child process 1045, status = succeeded4536worker1 # [ 66.823066] nix-daemon[834]: reaped child process 1047, status = succeeded4537worker1 # [ 67.103604] nix-grpc-daemon[768]: ts=2026-09-22T10:59:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479944538worker2 # [ 67.168293] nix-grpc-daemon[769]: ts=2026-09-22T10:59:32Z level=debug event=want client=7 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04539worker2 # [ 67.168589] nix-grpc-daemon[769]: ts=2026-09-22T10:59:32Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14540worker2 # [ 67.181193] nix-grpc-daemon[769]: ts=2026-09-22T10:59:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024541worker2 # [ 67.198337] nix-grpc-daemon[769]: ts=2026-09-22T10:59:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024542worker2 # [ 67.213317] nix-grpc-daemon[769]: ts=2026-09-22T10:59:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024543worker2 # [ 67.228447] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4544worker2 # [ 67.400411] nix-daemon[855]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4545worker2 # [ 67.420973] nix-daemon[855]: reaped child process 987, status = succeeded4546worker2 # [ 67.528206] nix-daemon[855]: accepted connection from pid 989, user nix-grpc-daemon (trusted)4547worker2 # [ 67.550479] nix-daemon[855]: reaped child process 992, status = succeeded4548lb # [ 67.530312] niks3-server[998]: 2026/09/22 10:59:33 INFO Received uploads request method=POST path=/api/pending_closures4549worker2 # [ 67.559816] nix-grpc-daemon[907]: time=2026-09-22T10:59:33.158Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4550worker2 # [ 67.562564] nix-grpc-daemon[907]: time=2026-09-22T10:59:33.158Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4551lb # [ 67.563727] niks3-server[998]: 2026/09/22 10:59:33 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4552worker2 # [ 67.612926] nix-grpc-daemon[907]: time=2026-09-22T10:59:33.211Z level=INFO msg="Uploading 1 narinfos"4553lb # [ 67.584316] niks3-server[998]: 2026/09/22 10:59:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4554lb # [ 67.595556] niks3-server[998]: 2026/09/22 10:59:33 INFO Signed narinfos id=12 count=14555lb # [ 67.602935] niks3-server[998]: 2026/09/22 10:59:33 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4556lb # [ 67.616767] niks3-server[998]: 2026/09/22 10:59:33 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4557lb # [ 67.624476] niks3-server[998]: 2026/09/22 10:59:33 INFO Completed upload id=124558worker2 # [ 67.645453] nix-grpc-daemon[907]: time=2026-09-22T10:59:33.244Z level=INFO msg="Upload complete. (398ms)"4559lb # [ 67.631582] niks3-server[998]: 2026/09/22 10:59:33 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4560worker2 # [ 67.656370] nix-grpc-daemon[769]: ts=2026-09-22T10:59:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=4244561worker2 # [ 67.674040] nix-grpc-daemon[769]: ts=2026-09-22T10:59:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024562worker2 # [ 67.681377] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4563worker2 # [ 67.696849] nix-daemon[855]: reaped child process 982, status = succeeded4564worker2 # [ 67.717933] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4565lb # [ 68.788768] envoy[898]: conn peer=[2001:db8:1::1]:47672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1429 ms=47614566lb # [ 68.789012] envoy[898]: conn peer=[2001:db8:1::1]:47684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1085 ms=36744567lb # [ 68.789172] envoy[898]: conn peer=[2001:db8:1::1]:47700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3001 tx=952 ms=66264568worker2 # [ 73.736705] nix-grpc-daemon[769]: ts=2026-09-22T10:59:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=7 outputs=04569worker2 # [ 73.774426] nix-daemon[855]: reaped child process 996, status = succeeded4570worker2 # [ 73.776847] nix-daemon[855]: reaped child process 994, status = succeeded4571(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.87 seconds)4572subtest: the first client leaving does not fail the second4573client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4574client # [ 73.948903] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4575client # Running as unit: share1.service; invocation ID: ba3b5935a2174c8da2bd5cb06506d0044576client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4577client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4578worker1 # [ 74.157235] nix-grpc-daemon[768]: ts=2026-09-22T10:59:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479944579client # [ 74.162967] nix[1017]: don't know how to build these paths:4580client # [ 74.163943] nix[1017]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4581worker2 # [ 74.220306] nix-grpc-daemon[769]: ts=2026-09-22T10:59:39Z level=debug event=want client=8 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04582worker1 # [ 74.181035] nix-grpc-daemon[768]: ts=2026-09-22T10:59:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944583worker2 # [ 74.226213] nix-grpc-daemon[769]: ts=2026-09-22T10:59:39Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14584worker1 # [ 74.198363] nix-grpc-daemon[768]: ts=2026-09-22T10:59:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944585client # [ 74.208950] nix[1017]: copying 1 paths to 'lb:50051'...4586client # [ 74.211992] nix[1017]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4587worker1 # [ 74.215419] nix-grpc-daemon[768]: ts=2026-09-22T10:59:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479944588worker1 # [ 74.230601] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4589worker1 # [ 74.401881] nix-daemon[834]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4590worker1 # [ 74.423305] nix-daemon[834]: reaped child process 1073, status = succeeded4591worker1 # [ 74.527833] nix-daemon[834]: accepted connection from pid 1075, user nix-grpc-daemon (trusted)4592worker1 # [ 74.545554] nix-daemon[834]: reaped child process 1078, status = succeeded4593lb # [ 74.573014] niks3-server[998]: 2026/09/22 10:59:40 INFO Received uploads request method=POST path=/api/pending_closures4594worker1 # [ 74.564935] nix-grpc-daemon[847]: time=2026-09-22T10:59:40.159Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4595worker1 # [ 74.566418] nix-grpc-daemon[847]: time=2026-09-22T10:59:40.159Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4596lb # [ 74.612140] niks3-server[998]: 2026/09/22 10:59:40 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4597worker1 # [ 74.604595] nix-grpc-daemon[847]: time=2026-09-22T10:59:40.199Z level=INFO msg="Uploading 1 narinfos"4598lb # [ 74.627261] niks3-server[998]: 2026/09/22 10:59:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4599lb # [ 74.637491] niks3-server[998]: 2026/09/22 10:59:40 INFO Signed narinfos id=13 count=14600lb # [ 74.645032] niks3-server[998]: 2026/09/22 10:59:40 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4601worker1 # [ 74.635286] nix-grpc-daemon[847]: time=2026-09-22T10:59:40.230Z level=INFO msg="Upload complete. (386ms)"4602lb # [ 74.656872] niks3-server[998]: 2026/09/22 10:59:40 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4603lb # [ 74.668445] niks3-server[998]: 2026/09/22 10:59:40 INFO Completed upload id=134604worker1 # [ 74.645243] nix-grpc-daemon[768]: ts=2026-09-22T10:59:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_in=4324605lb # [ 74.671849] niks3-server[998]: 2026/09/22 10:59:40 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4606worker1 # [ 74.660539] nix-grpc-daemon[768]: ts=2026-09-22T10:59:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944607worker1 # [ 74.667088] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4608worker1 # [ 74.671966] nix-daemon[834]: reaped child process 1068, status = succeeded4609client # [ 74.680546] nix[1017]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4610worker1 # [ 74.686017] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4611client # [ 75.050724] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4612client # Running as unit: share2.service; invocation ID: cd1ac9a4a2354f289988cf8eebddaa3f4613client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4614client # [ 75.266796] nix[1033]: don't know how to build these paths:4615client # [ 75.267810] nix[1033]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4616worker2 # [ 75.310544] nix-grpc-daemon[769]: ts=2026-09-22T10:59:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362024617worker2 # [ 75.323267] nix-grpc-daemon[769]: ts=2026-09-22T10:59:40Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04618worker1 # [ 75.286312] nix-grpc-daemon[768]: ts=2026-09-22T10:59:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944619client # [ 75.291212] nix[1033]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4620client: must succeed: sleep 2; systemctl kill -s INT share14621client # [ 78.435970] systemd[1]: share1.service: Sent signal SIGINT to main process 1017 (nix) on client request.4622client # [ 78.446004] nix[1017]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4623client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4624client: waiting for success: ! systemctl is-active share14625client # [ 78.461824] nix[1017]: error: interrupted by the user4626client # [ 78.518745] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4627client # [ 78.521334] systemd[1]: share1.service: Failed with result 'exit-code'.4628client # [ 78.524425] systemd[1]: share1.service: Consumed 175ms CPU time over 4.572s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8.1K outgoing IP traffic.4629lb # [ 78.793124] envoy[898]: conn peer=[2001:db8:1::1]:60898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2976 tx=978 ms=66174630lb # [ 78.798149] envoy[898]: conn peer=[2001:db8:1::1]:42416 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3003 tx=811 ms=43114631client: (finished: waiting for success: ! systemctl is-active share1, in 1.09 seconds)4632client: waiting for success: ! systemctl is-active share24633worker1 # [ 79.997217] nix-daemon[834]: reaped child process 1082, status = succeeded4634worker1 # [ 80.068083] nix-daemon[834]: accepted connection from pid 1116, user nix-grpc-daemon (trusted)4635worker1 # [ 80.090516] nix-daemon[834]: reaped child process 1119, status = succeeded4636lb # [ 80.111607] niks3-server[998]: 2026/09/22 10:59:45 INFO Received uploads request method=POST path=/api/pending_closures4637worker1 # [ 80.099840] nix-grpc-daemon[847]: time=2026-09-22T10:59:45.694Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4638worker1 # [ 80.104134] nix-grpc-daemon[847]: time=2026-09-22T10:59:45.697Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4639lb # [ 80.157161] niks3-server[998]: 2026/09/22 10:59:45 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4640lb # [ 80.168322] niks3-server[998]: 2026/09/22 10:59:45 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4641worker1 # [ 80.162222] nix-grpc-daemon[847]: time=2026-09-22T10:59:45.756Z level=INFO msg="Uploading 1 narinfos"4642lb # [ 80.182776] niks3-server[998]: 2026/09/22 10:59:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4643lb # [ 80.191597] niks3-server[998]: 2026/09/22 10:59:45 INFO Signed narinfos id=14 count=14644lb # [ 80.196211] niks3-server[998]: 2026/09/22 10:59:45 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4645lb # [ 80.205684] niks3-server[998]: 2026/09/22 10:59:45 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4646lb # [ 80.210836] niks3-server[998]: 2026/09/22 10:59:45 INFO Completed upload id=144647lb # [ 80.217056] niks3-server[998]: 2026/09/22 10:59:45 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4648worker1 # [ 80.189152] nix-grpc-daemon[847]: time=2026-09-22T10:59:45.784Z level=INFO msg="Upload complete. (288ms)"4649worker1 # [ 80.205417] nix-grpc-daemon[768]: ts=2026-09-22T10:59:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=8 outputs=14650worker1 # [ 80.220458] nix-grpc-daemon[768]: ts=2026-09-22T10:59:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=8 outputs=14651worker1 # [ 80.228524] nix-daemon[834]: reaped child process 1080, status = succeeded4652client # [ 80.236931] systemd[1]: share2.service: Deactivated successfully.4653client # [ 80.241265] systemd[1]: share2.service: Consumed 190ms CPU time over 5.190s wall clock time, 35.3M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4654client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4655client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4656client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4657(finished: subtest: the first client leaving does not fail the second, in 7.09 seconds)4658subtest: build hook: nix-daemon with builders = grpc://lb4659client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4660client # [ 81.107149] systemd[1]: Started Nix Daemon.4661client # [ 81.224947] nix-daemon[1085]: accepted connection from pid 1082, user root (trusted)4662worker2 # [ 81.417796] nix-grpc-daemon[769]: ts=2026-09-22T10:59:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:362024663worker1 # [ 81.379838] nix-grpc-daemon[768]: ts=2026-09-22T10:59:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944664worker2 # [ 81.440973] nix-grpc-daemon[769]: ts=2026-09-22T10:59:47Z level=debug event=want client=10 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04665worker2 # [ 81.448431] nix-grpc-daemon[769]: ts=2026-09-22T10:59:47Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14666worker2 # [ 81.457964] nix-grpc-daemon[769]: ts=2026-09-22T10:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024667worker2 # [ 81.469081] nix-grpc-daemon[769]: ts=2026-09-22T10:59:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024668worker2 # [ 81.483371] nix-grpc-daemon[769]: ts=2026-09-22T10:59:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024669worker2 # [ 81.498526] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4670worker2 # [ 81.653377] nix-daemon[855]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4671worker2 # [ 81.670155] nix-daemon[855]: reaped child process 1044, status = succeeded4672worker2 # [ 81.772691] nix-daemon[855]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4673worker2 # [ 81.791958] nix-daemon[855]: reaped child process 1049, status = succeeded4674lb # [ 81.773566] niks3-server[998]: 2026/09/22 10:59:47 INFO Received uploads request method=POST path=/api/pending_closures4675worker2 # [ 81.803619] nix-grpc-daemon[907]: time=2026-09-22T10:59:47.402Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4676worker2 # [ 81.805242] nix-grpc-daemon[907]: time=2026-09-22T10:59:47.402Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4677lb # [ 81.799348] niks3-server[998]: 2026/09/22 10:59:47 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4678lb # [ 81.815742] niks3-server[998]: 2026/09/22 10:59:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4679worker2 # [ 81.843075] nix-grpc-daemon[907]: time=2026-09-22T10:59:47.442Z level=INFO msg="Uploading 1 narinfos"4680lb # [ 81.826330] niks3-server[998]: 2026/09/22 10:59:47 INFO Signed narinfos id=15 count=14681lb # [ 81.833465] niks3-server[998]: 2026/09/22 10:59:47 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4682lb # [ 81.843957] niks3-server[998]: 2026/09/22 10:59:47 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4683lb # [ 81.852296] niks3-server[998]: 2026/09/22 10:59:47 INFO Completed upload id=154684lb # [ 81.855390] niks3-server[998]: 2026/09/22 10:59:47 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4685worker2 # [ 81.873888] nix-grpc-daemon[907]: time=2026-09-22T10:59:47.473Z level=INFO msg="Upload complete. (359ms)"4686worker2 # [ 81.884432] nix-grpc-daemon[769]: ts=2026-09-22T10:59:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=5124687worker2 # [ 81.898165] nix-grpc-daemon[769]: ts=2026-09-22T10:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024688worker2 # [ 81.906497] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4689worker2 # [ 81.909497] nix-daemon[855]: reaped child process 1039, status = succeeded4690worker2 # [ 81.921551] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4691worker2 # [ 82.967897] nix-daemon[855]: reaped child process 1053, status = succeeded4692worker2 # [ 83.078073] nix-daemon[855]: accepted connection from pid 1069, user nix-grpc-daemon (trusted)4693worker2 # [ 83.099499] nix-daemon[855]: reaped child process 1072, status = succeeded4694lb # [ 83.082830] niks3-server[998]: 2026/09/22 10:59:48 INFO Received uploads request method=POST path=/api/pending_closures4695worker2 # [ 83.114658] nix-grpc-daemon[907]: time=2026-09-22T10:59:48.712Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4696worker2 # [ 83.116232] nix-grpc-daemon[907]: time=2026-09-22T10:59:48.712Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4697lb # [ 83.119449] niks3-server[998]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4698lb # [ 83.132379] niks3-server[998]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4699lb # [ 83.141758] niks3-server[998]: 2026/09/22 10:59:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4700worker2 # [ 83.169618] nix-grpc-daemon[907]: time=2026-09-22T10:59:48.768Z level=INFO msg="Uploading 1 narinfos"4701lb # [ 83.151905] niks3-server[998]: 2026/09/22 10:59:48 INFO Signed narinfos id=16 count=14702lb # [ 83.158845] niks3-server[998]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4703lb # [ 83.171222] niks3-server[998]: 2026/09/22 10:59:48 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4704worker2 # [ 83.196933] nix-grpc-daemon[907]: time=2026-09-22T10:59:48.796Z level=INFO msg="Upload complete. (265ms)"4705lb # [ 83.179893] niks3-server[998]: 2026/09/22 10:59:48 INFO Completed upload id=164706lb # [ 83.183068] niks3-server[998]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4707worker2 # [ 83.210496] nix-grpc-daemon[769]: ts=2026-09-22T10:59:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=9 outputs=14708worker1 # [ 83.173155] nix-grpc-daemon[768]: ts=2026-09-22T10:59:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479944709worker2 # [ 83.229777] nix-daemon[855]: reaped child process 1051, status = succeeded4710worker1 # [ 83.186971] nix-grpc-daemon[768]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4711worker1 # [ 83.232114] nix-grpc-daemon[768]: ts=2026-09-22T10:59:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:479944712worker1 # [ 83.238273] nix-grpc-daemon[768]: ts=2026-09-22T10:59:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_out=1284713worker1 # [ 83.514735] nix-grpc-daemon[768]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:479944714worker2 # [ 83.562071] nix-grpc-daemon[769]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024715worker2 # [ 83.572086] nix-grpc-daemon[907]: time=2026-09-22T10:59:49.171Z level=INFO msg="All 1 paths already cached"4716worker2 # [ 83.589489] nix-grpc-daemon[769]: ts=2026-09-22T10:59:49Z level=debug event=want client=11 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04717worker2 # [ 83.596571] nix-grpc-daemon[769]: ts=2026-09-22T10:59:49Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14718worker2 # [ 83.638066] nix-grpc-daemon[769]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024719worker2 # [ 83.653608] nix-grpc-daemon[769]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024720worker2 # [ 83.668323] nix-grpc-daemon[907]: time=2026-09-22T10:59:49.267Z level=INFO msg="All 1 paths already cached"4721worker2 # [ 83.674811] nix-grpc-daemon[769]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024722worker2 # [ 83.689345] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4723worker2 # [ 83.844126] nix-daemon[855]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4724worker2 # [ 83.862083] nix-daemon[855]: reaped child process 1081, status = succeeded4725worker2 # [ 83.962055] nix-daemon[855]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)4726worker2 # [ 83.979069] nix-daemon[855]: reaped child process 1086, status = succeeded4727lb # [ 83.962641] niks3-server[998]: 2026/09/22 10:59:49 INFO Received uploads request method=POST path=/api/pending_closures4728worker2 # [ 83.993554] nix-grpc-daemon[907]: time=2026-09-22T10:59:49.592Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4729worker2 # [ 83.995052] nix-grpc-daemon[907]: time=2026-09-22T10:59:49.592Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4730lb # [ 83.995268] niks3-server[998]: 2026/09/22 10:59:49 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4731worker2 # [ 84.026925] nix-grpc-daemon[907]: time=2026-09-22T10:59:49.626Z level=INFO msg="Uploading 1 narinfos"4732lb # [ 84.010825] niks3-server[998]: 2026/09/22 10:59:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4733lb # [ 84.020926] niks3-server[998]: 2026/09/22 10:59:49 INFO Signed narinfos id=17 count=14734worker2 # [ 84.051553] nix-grpc-daemon[907]: time=2026-09-22T10:59:49.650Z level=INFO msg="Upload complete. (345ms)"4735lb # [ 84.030000] niks3-server[998]: 2026/09/22 10:59:49 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4736lb # [ 84.037390] niks3-server[998]: 2026/09/22 10:59:49 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4737lb # [ 84.042036] niks3-server[998]: 2026/09/22 10:59:49 INFO Completed upload id=174738worker2 # [ 84.061171] nix-grpc-daemon[769]: ts=2026-09-22T10:59:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=6244739lb # [ 84.044252] niks3-server[998]: 2026/09/22 10:59:49 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4740worker2 # [ 84.071076] nix-grpc-daemon[769]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024741worker2 # [ 84.077177] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4742worker2 # [ 84.081638] nix-daemon[855]: reaped child process 1076, status = succeeded4743worker2 # [ 84.093545] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4744worker2 # [ 85.176710] nix-daemon[855]: reaped child process 1090, status = succeeded4745worker2 # [ 85.272507] nix-daemon[855]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)4746worker2 # [ 85.290038] nix-daemon[855]: reaped child process 1109, status = succeeded4747lb # [ 85.277850] niks3-server[998]: 2026/09/22 10:59:50 INFO Received uploads request method=POST path=/api/pending_closures4748worker2 # [ 85.308421] nix-grpc-daemon[907]: time=2026-09-22T10:59:50.906Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4749worker2 # [ 85.310151] nix-grpc-daemon[907]: time=2026-09-22T10:59:50.906Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4750lb # [ 85.327256] niks3-server[998]: 2026/09/22 10:59:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4751worker2 # [ 85.359713] nix-grpc-daemon[907]: time=2026-09-22T10:59:50.958Z level=INFO msg="Uploading 1 narinfos"4752lb # [ 85.341791] niks3-server[998]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4753lb # [ 85.351927] niks3-server[998]: 2026/09/22 10:59:50 INFO Signed narinfos id=18 count=14754lb # [ 85.359349] niks3-server[998]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4755worker2 # [ 85.380626] nix-grpc-daemon[907]: time=2026-09-22T10:59:50.979Z level=INFO msg="Upload complete. (274ms)"4756lb # [ 85.366257] niks3-server[998]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4757lb # [ 85.370151] niks3-server[998]: 2026/09/22 10:59:50 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4758lb # [ 85.374175] niks3-server[998]: 2026/09/22 10:59:50 INFO Completed upload id=184759worker2 # [ 85.393509] nix-grpc-daemon[769]: ts=2026-09-22T10:59:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=10 outputs=14760lb # [ 85.376251] niks3-server[998]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4761worker1 # [ 85.362254] nix-grpc-daemon[768]: ts=2026-09-22T10:59:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479944762worker2 # [ 85.410349] nix-daemon[855]: reaped child process 1088, status = succeeded4763worker1 # [ 85.376776] nix-grpc-daemon[768]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4764worker2 # [ 85.460112] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:362024765worker2 # [ 85.466100] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_out=1764766worker2 # [ 85.704295] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:362024767worker1 # [ 85.666354] nix-grpc-daemon[768]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944768worker1 # [ 85.678250] nix-grpc-daemon[847]: time=2026-09-22T10:59:51.272Z level=INFO msg="All 2 paths already cached"4769worker2 # [ 85.739004] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=debug event=want client=12 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04770worker2 # [ 85.746414] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14771worker2 # [ 85.755005] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024772worker2 # [ 85.766642] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024773worker2 # [ 85.781183] nix-grpc-daemon[907]: time=2026-09-22T10:59:51.380Z level=INFO msg="All 2 paths already cached"4774worker2 # [ 85.787692] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024775worker2 # [ 85.801822] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4776worker2 # [ 85.959458] nix-daemon[855]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4777worker2 # [ 85.977527] nix-daemon[855]: reaped child process 1118, status = succeeded4778worker2 # [ 86.078552] nix-daemon[855]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)4779worker2 # [ 86.097152] nix-daemon[855]: reaped child process 1123, status = succeeded4780lb # [ 86.079974] niks3-server[998]: 2026/09/22 10:59:51 INFO Received uploads request method=POST path=/api/pending_closures4781worker2 # [ 86.112769] nix-grpc-daemon[907]: time=2026-09-22T10:59:51.711Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4782worker2 # [ 86.114277] nix-grpc-daemon[907]: time=2026-09-22T10:59:51.711Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4783lb # [ 86.111382] niks3-server[998]: 2026/09/22 10:59:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4784worker2 # [ 86.137527] nix-grpc-daemon[907]: time=2026-09-22T10:59:51.736Z level=INFO msg="Uploading 1 narinfos"4785lb # [ 86.119292] niks3-server[998]: 2026/09/22 10:59:51 INFO Signed narinfos id=19 count=14786lb # [ 86.126827] niks3-server[998]: 2026/09/22 10:59:51 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4787lb # [ 86.133913] niks3-server[998]: 2026/09/22 10:59:51 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4788lb # [ 86.142189] niks3-server[998]: 2026/09/22 10:59:51 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4789lb # [ 86.147628] niks3-server[998]: 2026/09/22 10:59:51 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4790lb # [ 86.152755] niks3-server[998]: 2026/09/22 10:59:51 INFO Completed upload id=194791worker2 # [ 86.174044] nix-grpc-daemon[907]: time=2026-09-22T10:59:51.773Z level=INFO msg="Upload complete. (355ms)"4792worker2 # [ 86.180317] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=7524793worker2 # [ 86.194503] nix-grpc-daemon[769]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024794worker2 # [ 86.203370] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4795worker2 # [ 86.206368] nix-daemon[855]: reaped child process 1113, status = succeeded4796worker2 # [ 86.218511] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4797worker2 # [ 87.685389] nix-daemon[855]: reaped child process 1127, status = succeeded4798worker2 # [ 87.799854] nix-daemon[855]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4799worker2 # [ 87.819951] nix-daemon[855]: reaped child process 1146, status = succeeded4800lb # [ 87.806283] niks3-server[998]: 2026/09/22 10:59:53 INFO Received uploads request method=POST path=/api/pending_closures4801worker2 # [ 87.836423] nix-grpc-daemon[907]: time=2026-09-22T10:59:53.434Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4802worker2 # [ 87.838107] nix-grpc-daemon[907]: time=2026-09-22T10:59:53.434Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4803lb # [ 87.854156] niks3-server[998]: 2026/09/22 10:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4804worker2 # [ 87.885683] nix-grpc-daemon[907]: time=2026-09-22T10:59:53.484Z level=INFO msg="Uploading 1 narinfos"4805lb # [ 87.870216] niks3-server[998]: 2026/09/22 10:59:53 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4806lb # [ 87.878031] niks3-server[998]: 2026/09/22 10:59:53 INFO Signed narinfos id=20 count=14807lb # [ 87.882453] niks3-server[998]: 2026/09/22 10:59:53 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4808lb # [ 87.890261] niks3-server[998]: 2026/09/22 10:59:53 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4809lb # [ 87.893405] niks3-server[998]: 2026/09/22 10:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4810lb # [ 87.898470] niks3-server[998]: 2026/09/22 10:59:53 INFO Completed upload id=204811worker2 # [ 87.918602] nix-grpc-daemon[907]: time=2026-09-22T10:59:53.517Z level=INFO msg="Upload complete. (289ms)"4812lb # [ 87.900565] niks3-server[998]: 2026/09/22 10:59:53 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4813worker2 # [ 87.934377] nix-grpc-daemon[769]: ts=2026-09-22T10:59:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=11 outputs=14814worker2 # [ 87.950402] nix-grpc-daemon[769]: ts=2026-09-22T10:59:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:362024815worker2 # [ 87.955857] nix-daemon[855]: reaped child process 1125, status = succeeded4816worker1 # [ 87.915132] nix-grpc-daemon[768]: ts=2026-09-22T10:59:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:479944817worker1 # [ 87.932471] nix-grpc-daemon[768]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4818worker1 # [ 87.948786] nix-grpc-daemon[768]: ts=2026-09-22T10:59:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_out=2324819client # [ 88.080413] nix-daemon[1085]: reaped child process 1087, status = succeeded4820client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.13 seconds)4821client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4822client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4823client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4824client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4825client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4826client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4827(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.25 seconds)4828subtest: an input only one worker has still reaches the builder4829client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4830client # this derivation will be built:4831client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4832client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4833client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)4834client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4835client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4836worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4837lb # [ 88.791029] envoy[898]: conn peer=[2001:db8:1::1]:42428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1085 ms=49514838worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4839worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4840worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4841client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24842client # [ 89.064929] nix-daemon[1085]: accepted connection from pid 1168, user root (trusted)4843client # this derivation will be built:4844client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4845client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4846worker2 # [ 89.257163] nix-grpc-daemon[769]: ts=2026-09-22T10:59:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:362024847worker1 # [ 89.218982] nix-grpc-daemon[768]: ts=2026-09-22T10:59:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944848worker1 # [ 89.396034] nix-daemon[834]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)4849lb # [ 88.796109] envoy[898]: conn peer=[2001:db8:1::1]:47316 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3427 tx=1855 ms=19224850lb # [ 88.796282] envoy[898]: conn peer=[2001:db8:1::1]:47324 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=734851lb # [ 88.796399] envoy[898]: conn peer=[2001:db8:1::1]:47336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3901 tx=2046 ms=19254852worker1 # [ 89.414257] nix-daemon[834]: reaped child process 1132, status = succeeded4853lb # [ 88.796493] envoy[898]: conn peer=[2001:db8:1::1]:47348 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=294854lb # [ 88.796600] envoy[898]: conn peer=[2001:db8:1::1]:47364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2167 ms=23474855lb # [ 88.796712] envoy[898]: conn peer=[2001:db8:1::1]:47372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=974856lb # [ 89.450592] niks3-server[998]: 2026/09/22 10:59:55 INFO Received uploads request method=POST path=/api/pending_closures4857worker1 # [ 89.438866] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.033Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4858worker1 # [ 89.440997] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.033Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4859lb # [ 89.479354] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4860worker1 # [ 89.471369] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.066Z level=INFO msg="Uploading 1 narinfos"4861lb # [ 89.493888] niks3-server[998]: 2026/09/22 10:59:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4862lb # [ 89.503730] niks3-server[998]: 2026/09/22 10:59:55 INFO Signed narinfos id=21 count=14863lb # [ 89.510193] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4864lb # [ 89.520298] niks3-server[998]: 2026/09/22 10:59:55 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4865worker1 # [ 89.499472] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.094Z level=INFO msg="Upload complete. (274ms)"4866lb # [ 89.529617] niks3-server[998]: 2026/09/22 10:59:55 INFO Completed upload id=214867lb # [ 89.532214] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4868worker2 # [ 89.564422] nix-grpc-daemon[769]: ts=2026-09-22T10:59:55Z level=debug event=want client=13 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04869worker2 # [ 89.571219] nix-grpc-daemon[769]: ts=2026-09-22T10:59:55Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14870worker1 # [ 89.575714] nix-grpc-daemon[768]: ts=2026-09-22T10:59:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944871worker1 # [ 89.591036] nix-grpc-daemon[768]: ts=2026-09-22T10:59:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944872client # copying 2 paths to 'lb:50051'...4873client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4874worker1 # [ 89.609687] nix-grpc-daemon[768]: ts=2026-09-22T10:59:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479944875client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4876worker1 # [ 89.625301] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4877worker1 # [ 89.790502] nix-daemon[834]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)4878worker1 # [ 89.809901] nix-daemon[834]: reaped child process 1139, status = succeeded4879worker1 # [ 89.912939] nix-daemon[834]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)4880worker1 # [ 89.931335] nix-daemon[834]: reaped child process 1144, status = succeeded4881lb # [ 89.955937] niks3-server[998]: 2026/09/22 10:59:55 INFO Received uploads request method=POST path=/api/pending_closures4882lb # [ 89.965158] niks3-server[998]: 2026/09/22 10:59:55 INFO Received uploads request method=POST path=/api/pending_closures4883worker1 # [ 89.952774] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.547Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4884worker1 # [ 89.954287] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.547Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4885worker1 # [ 89.956083] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.547Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4886lb # [ 90.001318] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4887lb # [ 90.014870] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4888lb # [ 90.026629] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4889lb # [ 90.036248] niks3-server[998]: 2026/09/22 10:59:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4890lb # [ 90.043676] niks3-server[998]: 2026/09/22 10:59:55 INFO Signed narinfos id=22 count=14891worker1 # [ 90.021834] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.616Z level=INFO msg="Uploading 3 narinfos"4892lb # [ 90.048598] niks3-server[998]: 2026/09/22 10:59:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4893lb # [ 90.054531] niks3-server[998]: 2026/09/22 10:59:55 INFO Signed narinfos id=23 count=24894lb # [ 90.058847] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4895lb # [ 90.067967] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4896lb # [ 90.078482] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4897lb # [ 90.085258] niks3-server[998]: 2026/09/22 10:59:55 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4898lb # [ 90.092667] niks3-server[998]: 2026/09/22 10:59:55 INFO Completed upload id=234899lb # [ 90.096993] niks3-server[998]: 2026/09/22 10:59:55 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4900worker1 # [ 90.076040] nix-grpc-daemon[847]: time=2026-09-22T10:59:55.671Z level=INFO msg="Upload complete. (431ms)"4901lb # [ 90.099630] niks3-server[998]: 2026/09/22 10:59:55 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4902lb # [ 90.103483] niks3-server[998]: 2026/09/22 10:59:55 INFO Completed upload id=224903worker1 # [ 90.088158] nix-grpc-daemon[768]: ts=2026-09-22T10:59:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=2 nar_bytes_in=9844904worker1 # [ 90.104803] nix-grpc-daemon[768]: ts=2026-09-22T10:59:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479944905worker1 # [ 90.114121] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4906worker1 # [ 90.121719] nix-daemon[834]: reaped child process 1134, status = succeeded4907client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4908worker1 # [ 90.135877] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4909client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4910client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4911worker1 # [ 91.379327] nix-daemon[834]: reaped child process 1148, status = succeeded4912worker1 # [ 91.479968] nix-daemon[834]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)4913worker1 # [ 91.497740] nix-daemon[834]: reaped child process 1167, status = succeeded4914lb # [ 91.526953] niks3-server[998]: 2026/09/22 10:59:57 INFO Received uploads request method=POST path=/api/pending_closures4915worker1 # [ 91.516424] nix-grpc-daemon[847]: time=2026-09-22T10:59:57.110Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4916lb # [ 91.560241] niks3-server[998]: 2026/09/22 10:59:57 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4917worker1 # [ 91.549774] nix-grpc-daemon[847]: time=2026-09-22T10:59:57.144Z level=INFO msg="Uploading 1 narinfos"4918lb # [ 91.572950] niks3-server[998]: 2026/09/22 10:59:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4919lb # [ 91.584220] niks3-server[998]: 2026/09/22 10:59:57 INFO Signed narinfos id=24 count=14920lb # [ 91.588909] niks3-server[998]: 2026/09/22 10:59:57 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4921lb # [ 91.596948] niks3-server[998]: 2026/09/22 10:59:57 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4922worker1 # [ 91.581816] nix-grpc-daemon[847]: time=2026-09-22T10:59:57.177Z level=INFO msg="Upload complete. (231ms)"4923lb # [ 91.604759] niks3-server[998]: 2026/09/22 10:59:57 INFO Completed upload id=244924lb # [ 91.606905] niks3-server[998]: 2026/09/22 10:59:57 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4925client # copying 1 paths...4926worker1 # [ 91.592925] nix-grpc-daemon[768]: ts=2026-09-22T10:59:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=12 outputs=14927worker1 # [ 91.607009] nix-grpc-daemon[768]: ts=2026-09-22T10:59:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479944928client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4929worker1 # [ 91.611413] nix-daemon[834]: reaped child process 1146, status = succeeded4930worker1 # [ 91.634388] nix-grpc-daemon[768]: ts=2026-09-22T10:59:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:479944931worker1 # [ 91.639723] nix-grpc-daemon[768]: ts=2026-09-22T10:59:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_out=1204932client # [ 91.762496] nix-daemon[1085]: reaped child process 1171, status = succeeded4933client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.90 seconds)4934client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24935client # [ 91.934162] nix-daemon[1085]: accepted connection from pid 1190, user root (trusted)4936client # this derivation will be built:4937client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4938worker1 # [ 92.083170] nix-grpc-daemon[768]: ts=2026-09-22T10:59:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:479944939client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4940worker1 # [ 92.090809] nix-grpc-daemon[768]: ts=2026-09-22T10:59:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479944941worker1 # [ 92.097528] nix-grpc-daemon[847]: time=2026-09-22T10:59:57.691Z level=INFO msg="All 1 paths already cached"4942worker2 # [ 92.159631] nix-grpc-daemon[769]: ts=2026-09-22T10:59:57Z level=debug event=want client=14 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04943worker2 # [ 92.159931] nix-grpc-daemon[769]: ts=2026-09-22T10:59:57Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14944worker2 # [ 92.173468] nix-grpc-daemon[769]: ts=2026-09-22T10:59:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024945worker2 # [ 92.181041] nix-grpc-daemon[769]: ts=2026-09-22T10:59:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024946client # copying 1 paths to 'lb:50051'...4947client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4948worker2 # [ 92.214478] nix-grpc-daemon[769]: ts=2026-09-22T10:59:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024949worker2 # [ 92.229408] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4950worker2 # [ 92.252758] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5)4951worker2 # [ 92.261550] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 2/5)4952worker2 # [ 92.354641] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 203 ms (attempt 3/5)4953worker2 # [ 92.559825] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 335 ms (attempt 4/5)4954worker2 # [ 92.897645] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4955worker2 # [ 92.903231] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4956worker2 # [ 92.911254] nix-grpc-daemon[769]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4957worker2 # [ 93.090214] nix-daemon[855]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)4958worker2 # [ 93.110459] nix-daemon[855]: reaped child process 1165, status = succeeded4959worker2 # [ 93.214969] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)4960worker2 # [ 93.232529] nix-daemon[855]: reaped child process 1170, status = succeeded4961lb # [ 93.217089] niks3-server[998]: 2026/09/22 10:59:58 INFO Received uploads request method=POST path=/api/pending_closures4962worker2 # [ 93.248797] nix-grpc-daemon[907]: time=2026-09-22T10:59:58.847Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4963worker2 # [ 93.250236] nix-grpc-daemon[907]: time=2026-09-22T10:59:58.847Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4964lb # [ 93.248521] niks3-server[998]: 2026/09/22 10:59:58 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4965worker2 # [ 93.292352] nix-grpc-daemon[907]: time=2026-09-22T10:59:58.891Z level=INFO msg="Uploading 1 narinfos"4966lb # [ 93.270511] niks3-server[998]: 2026/09/22 10:59:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4967lb # [ 93.277944] niks3-server[998]: 2026/09/22 10:59:58 INFO Signed narinfos id=25 count=14968lb # [ 93.287566] niks3-server[998]: 2026/09/22 10:59:58 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4969lb # [ 93.299874] niks3-server[998]: 2026/09/22 10:59:58 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4970lb # [ 93.310037] niks3-server[998]: 2026/09/22 10:59:58 INFO Completed upload id=254971worker2 # [ 93.330368] nix-grpc-daemon[907]: time=2026-09-22T10:59:58.929Z level=INFO msg="Upload complete. (399ms)"4972lb # [ 93.317619] niks3-server[998]: 2026/09/22 10:59:58 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4973worker2 # [ 93.340992] nix-grpc-daemon[769]: ts=2026-09-22T10:59:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 paths=1 nar_bytes_in=5684974worker2 # [ 93.357136] nix-grpc-daemon[769]: ts=2026-09-22T10:59:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024975worker2 # [ 93.367941] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4976worker2 # [ 93.372490] nix-daemon[855]: reaped child process 1155, status = succeeded4977worker2 # [ 93.387753] nix-grpc-daemon[769]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4978client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4979client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4980worker2 # [ 93.408311] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4981client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4982worker2 # [ 94.476152] nix-daemon[855]: reaped child process 1175, status = succeeded4983worker2 # [ 94.564891] nix-daemon[855]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)4984worker2 # [ 94.583884] nix-daemon[855]: reaped child process 1194, status = succeeded4985lb # [ 94.569022] niks3-server[998]: 2026/09/22 11:00:00 INFO Received uploads request method=POST path=/api/pending_closures4986worker2 # [ 94.603010] nix-grpc-daemon[907]: time=2026-09-22T11:00:00.199Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4987lb # [ 94.602098] niks3-server[998]: 2026/09/22 11:00:00 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4988worker2 # [ 94.633832] nix-grpc-daemon[907]: time=2026-09-22T11:00:00.232Z level=INFO msg="Uploading 1 narinfos"4989lb # [ 94.614083] niks3-server[998]: 2026/09/22 11:00:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4990lb # [ 94.626671] niks3-server[998]: 2026/09/22 11:00:00 INFO Signed narinfos id=26 count=14991lb # [ 94.633176] niks3-server[998]: 2026/09/22 11:00:00 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4992lb # [ 94.641867] niks3-server[998]: 2026/09/22 11:00:00 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4993worker2 # [ 94.674011] nix-grpc-daemon[907]: time=2026-09-22T11:00:00.273Z level=INFO msg="Upload complete. (253ms)"4994lb # [ 94.654450] niks3-server[998]: 2026/09/22 11:00:00 INFO Completed upload id=264995lb # [ 94.659996] niks3-server[998]: 2026/09/22 11:00:00 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4996client # copying 1 paths...4997worker2 # [ 94.689162] nix-grpc-daemon[769]: ts=2026-09-22T11:00:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=13 outputs=14998worker1 # [ 94.650326] nix-grpc-daemon[768]: ts=2026-09-22T11:00:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479944999worker2 # [ 94.704936] nix-daemon[855]: reaped child process 1172, status = succeeded5000worker1 # [ 94.662724] nix-grpc-daemon[768]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5001client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5002worker2 # [ 94.749226] nix-grpc-daemon[769]: ts=2026-09-22T11:00:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:362025003worker2 # [ 94.756112] nix-grpc-daemon[769]: ts=2026-09-22T11:00:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_out=1205004client # [ 94.796333] nix-daemon[1085]: reaped child process 1193, status = succeeded5005client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.03 seconds)5006client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25007client # [ 94.968692] nix-daemon[1085]: accepted connection from pid 1212, user root (trusted)5008client # this derivation will be built:5009client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5010worker1 # [ 95.117408] nix-grpc-daemon[768]: ts=2026-09-22T11:00:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:479945011client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5012worker2 # [ 95.164803] nix-grpc-daemon[769]: ts=2026-09-22T11:00:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362025013worker2 # [ 95.176236] nix-grpc-daemon[907]: time=2026-09-22T11:00:00.775Z level=INFO msg="All 1 paths already cached"5014worker2 # [ 95.193852] nix-grpc-daemon[769]: ts=2026-09-22T11:00:00Z level=debug event=want client=15 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05015worker1 # [ 95.153993] nix-grpc-daemon[768]: ts=2026-09-22T11:00:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479945016worker2 # [ 95.200578] nix-grpc-daemon[769]: ts=2026-09-22T11:00:00Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15017worker1 # [ 95.170243] nix-grpc-daemon[768]: ts=2026-09-22T11:00:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479945018client # copying 1 paths to 'lb:50051'...5019worker1 # [ 95.186037] nix-grpc-daemon[847]: time=2026-09-22T11:00:00.779Z level=INFO msg="All 1 paths already cached"5020client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5021worker1 # [ 95.191484] nix-grpc-daemon[768]: ts=2026-09-22T11:00:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479945022worker1 # [ 95.206647] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5023worker1 # [ 95.362239] nix-daemon[834]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)5024worker1 # [ 95.382298] nix-daemon[834]: reaped child process 1175, status = succeeded5025worker1 # [ 95.486537] nix-daemon[834]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5026worker1 # [ 95.505370] nix-daemon[834]: reaped child process 1180, status = succeeded5027lb # [ 95.530384] niks3-server[998]: 2026/09/22 11:00:01 INFO Received uploads request method=POST path=/api/pending_closures5028worker1 # [ 95.520061] nix-grpc-daemon[847]: time=2026-09-22T11:00:01.115Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5029worker1 # [ 95.521556] nix-grpc-daemon[847]: time=2026-09-22T11:00:01.115Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5030lb # [ 95.562641] niks3-server[998]: 2026/09/22 11:00:01 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5031lb # [ 95.579209] niks3-server[998]: 2026/09/22 11:00:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5032worker1 # [ 95.567801] nix-grpc-daemon[847]: time=2026-09-22T11:00:01.162Z level=INFO msg="Uploading 1 narinfos"5033lb # [ 95.589248] niks3-server[998]: 2026/09/22 11:00:01 INFO Signed narinfos id=27 count=15034lb # [ 95.596464] niks3-server[998]: 2026/09/22 11:00:01 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5035lb # [ 95.607343] niks3-server[998]: 2026/09/22 11:00:01 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5036worker1 # [ 95.592657] nix-grpc-daemon[847]: time=2026-09-22T11:00:01.187Z level=INFO msg="Upload complete. (369ms)"5037lb # [ 95.617657] niks3-server[998]: 2026/09/22 11:00:01 INFO Completed upload id=275038worker1 # [ 95.600203] nix-grpc-daemon[768]: ts=2026-09-22T11:00:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=0 paths=1 nar_bytes_in=5685039lb # [ 95.622428] niks3-server[998]: 2026/09/22 11:00:01 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5040worker1 # [ 95.615464] nix-grpc-daemon[768]: ts=2026-09-22T11:00:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479945041worker1 # [ 95.622920] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5042worker1 # [ 95.628282] nix-daemon[834]: reaped child process 1170, status = succeeded5043client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5044worker1 # [ 95.643207] nix-daemon[834]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5045client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5046client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5047worker1 # [ 96.536715] nix-daemon[834]: reaped child process 1184, status = succeeded5048worker1 # [ 96.623870] nix-daemon[834]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)5049worker1 # [ 96.645479] nix-daemon[834]: reaped child process 1203, status = succeeded5050lb # [ 96.673881] niks3-server[998]: 2026/09/22 11:00:02 INFO Received uploads request method=POST path=/api/pending_closures5051worker1 # [ 96.662905] nix-grpc-daemon[847]: time=2026-09-22T11:00:02.255Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5052lb # [ 96.705126] niks3-server[998]: 2026/09/22 11:00:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5053worker1 # [ 96.691183] nix-grpc-daemon[847]: time=2026-09-22T11:00:02.286Z level=INFO msg="Uploading 1 narinfos"5054lb # [ 96.720259] niks3-server[998]: 2026/09/22 11:00:02 INFO Signed narinfos id=28 count=15055lb # [ 96.722966] niks3-server[998]: 2026/09/22 11:00:02 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5056lb # [ 96.731941] niks3-server[998]: 2026/09/22 11:00:02 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5057worker1 # [ 96.721531] nix-grpc-daemon[847]: time=2026-09-22T11:00:02.316Z level=INFO msg="Upload complete. (268ms)"5058lb # [ 96.745057] niks3-server[998]: 2026/09/22 11:00:02 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5059lb # [ 96.749475] niks3-server[998]: 2026/09/22 11:00:02 INFO Completed upload id=285060client # copying 1 paths...5061lb # [ 96.753264] niks3-server[998]: 2026/09/22 11:00:02 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5062worker1 # [ 96.733033] nix-grpc-daemon[768]: ts=2026-09-22T11:00:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47994 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=14 outputs=15063worker2 # [ 96.781179] nix-grpc-daemon[769]: ts=2026-09-22T11:00:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:362025064worker1 # [ 96.749151] nix-daemon[834]: reaped child process 1182, status = succeeded5065worker2 # [ 96.793641] nix-grpc-daemon[769]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5066client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5067worker2 # [ 96.838046] nix-grpc-daemon[769]: ts=2026-09-22T11:00:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:362025068worker2 # [ 96.843225] nix-grpc-daemon[769]: ts=2026-09-22T11:00:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_out=1205069client # [ 96.920809] nix-daemon[1085]: reaped child process 1215, status = succeeded5070client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.12 seconds)5071(finished: subtest: an input only one worker has still reaches the builder, in 8.71 seconds)5072subtest: upload whose reference the worker lacks is completed from the cache5073client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5074client # this derivation will be built:5075client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5076client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5077client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.34 seconds)5078client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5079client # this derivation will be built:5080client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5081client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5082client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.30 seconds)5083client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5084client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5085worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5086worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)5087worker2 # [ 97.984969] systemd[1]: Stopping Nix worker-protocol over gRPC...5088worker2 # [ 98.185221] nix-grpc-daemon[769]: ts=2026-09-22T11:00:03Z level=info event=signal_exit5089lb # [ 98.333459] niks3-server[998]: 2026/09/22 11:00:03 INFO lead: released remote=192.168.1.4:493325090lb # [ 98.790725] envoy[898]: conn peer=[2001:db8:1::1]:47386 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3879 tx=1857 ms=24785091lb # [ 98.795896] envoy[898]: conn peer=[2001:db8:1::1]:39016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=625092lb # [ 98.800863] envoy[898]: conn peer=[2001:db8:1::1]:39024 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2015 ms=26485093lb # [ 98.805623] envoy[898]: conn peer=[2001:db8:1::1]:39034 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=295094lb # [ 98.810251] envoy[898]: conn peer=[2001:db8:1::1]:39036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2012 ms=17385095lb # [ 98.815345] envoy[898]: conn peer=[2001:db8:1::1]:39048 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=675096worker1 # [ 99.315658] nix-grpc-daemon[768]: ts=2026-09-22T11:00:04Z level=info event=scheduler_restarting addr=lb:500515097worker2 # [ 99.360008] nix-grpc-daemon[769]: ts=2026-09-22T11:00:04Z level=info event=scheduler_restarting5098worker2 # [ 99.362471] nix-grpc-daemon[769]: ts=2026-09-22T11:00:04Z level=info event=scheduler_restarting addr=lb:500515099worker2 # [ 99.362668] nix-grpc-daemon[769]: ts=2026-09-22T11:00:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453245100worker2 # [ 99.362805] nix-grpc-daemon[769]: ts=2026-09-22T11:00:04Z level=info event=worker_gone addr=192.168.1.4:500515101worker2 # [ 99.362936] nix-grpc-daemon[769]: ts=2026-09-22T11:00:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453245102worker2 # [ 99.363043] nix-grpc-daemon[769]: ts=2026-09-22T11:00:04Z level=info event=worker_gone addr=192.168.1.3:500515103worker2 # [ 99.413951] nix-daemon[855]: reaped child process 857, status = succeeded5104worker1 # [ 99.515437] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_connected addr=lb:500515105worker1 # [ 99.520748] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5106worker2 # [ 99.579888] nix-grpc-daemon[769]: ts=2026-09-22T11:00:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5107worker1 # [ 99.721673] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_connected addr=lb:500515108worker1 # [ 99.726931] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5109worker1 # [ 99.927825] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_connected addr=lb:500515110worker1 # [ 99.933285] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5111worker2 # [ 100.132556] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5112worker2 # [ 100.140301] systemd[1]: Stopped Nix worker-protocol over gRPC.5113worker2 # [ 100.142855] systemd[1]: nix-grpc-daemon.service: Consumed 3.030s CPU time over 1min 21.197s wall clock time, 61.9M memory peak, 80.2K incoming IP traffic, 221.8K outgoing IP traffic.5114worker2 # [ 100.161416] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5115worker2 # [ 100.163432] systemd[1]: Closed Nix worker-protocol over gRPC.5116lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5117lb: must succeed: curl -sf localhost:9901/clusters5118worker2 # [ 100.172232] systemd[1]: nix-grpc-daemon.socket: Consumed 562us CPU time over 1min 30.058s wall clock time, 256K memory peak, 95.3K incoming IP traffic, 81K outgoing IP traffic.5119worker1 # [ 100.133961] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_connected addr=lb:500515120worker1 # [ 100.139278] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5121lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5122lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5123client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25124worker1 # [ 100.341240] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_connected addr=lb:500515125worker1 # [ 100.346018] nix-grpc-daemon[768]: ts=2026-09-22T11:00:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5126worker1 # [ 100.418997] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479945127worker1 # [ 100.428483] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479945128client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5129client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.26 seconds)5130worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5131worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5132worker1 # [ 100.542859] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_connected addr=lb:500515133worker1 # [ 100.548121] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5134worker2 # [ 100.598990] systemd[1]: Listening on Nix worker-protocol over gRPC.5135worker1 # [ 100.627305] systemd[1]: Stopping Nix worker-protocol over gRPC...5136worker1 # [ 100.749093] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_connected addr=lb:500515137worker1 # [ 100.754411] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5138worker1 # [ 100.955241] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_connected addr=lb:500515139worker1 # [ 100.960556] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5140worker1 # [ 101.039167] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=signal_exit5141worker1 # [ 101.161173] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_connected addr=lb:500515142worker1 # [ 101.166222] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5143worker2 # [ 101.329069] systemd[1]: Starting scheduler-token.service...5144worker1 # [ 101.370321] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_connected addr=lb:500515145worker2 # [ 101.415907] systemd[1]: scheduler-token.service: Deactivated successfully.5146worker1 # [ 101.374756] nix-grpc-daemon[768]: ts=2026-09-22T11:00:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5147worker2 # [ 101.420642] systemd[1]: Finished scheduler-token.service.5148worker2 # [ 101.429910] systemd[1]: scheduler-token.service: Consumed 57ms CPU time over 91ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5149worker2 # [ 101.453524] systemd[1]: Starting Nix worker-protocol over gRPC...5150worker1 # [ 101.571568] nix-grpc-daemon[768]: ts=2026-09-22T11:00:07Z level=info event=scheduler_connected addr=lb:500515151worker1 # [ 101.577501] nix-grpc-daemon[768]: ts=2026-09-22T11:00:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5152worker2 # [ 101.625326] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5153worker2 # [ 101.627645] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5154worker2 # [ 101.631580] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15155worker2 # [ 101.637487] nix-grpc-daemon[1221]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5156worker2 # [ 101.638956] nix-grpc-daemon[1221]: E0000 00:00:1790074807.235933 1221 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5157lb # [ 101.647177] niks3-server[998]: 2026/09/22 11:00:07 INFO lead: acquired remote=192.168.1.4:538145158worker2 # [ 101.680177] nix-grpc-daemon[1221]: E0000 00:00:1790074807.242226 1221 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5159worker2 # [ 101.682531] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5160worker2 # [ 101.708216] nix-grpc-daemon[1221]: E0000 00:00:1790074807.259977 1261 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5161worker2 # [ 101.711157] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=scheduler_take_over5162worker2 # [ 101.713165] systemd[1]: Started Nix worker-protocol over gRPC.5163worker2 # [ 101.713988] nix-grpc-daemon[1221]: E0000 00:00:1790074807.289615 1261 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5164worker2 # [ 101.716390] nix-grpc-daemon[1221]: E0000 00:00:1790074807.289824 1261 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5165worker2 # [ 101.722773] nix-grpc-daemon[1221]: E0000 00:00:1790074807.305741 1261 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5166worker2 # [ 101.727616] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=scheduler_connected addr=lb:500515167worker2 # [ 101.741822] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=oidc_jwks provider=mock keys=15168worker2 # [ 101.743206] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:505765169worker2 # [ 101.746563] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05170worker1 # [ 101.779439] nix-grpc-daemon[768]: ts=2026-09-22T11:00:07Z level=info event=scheduler_connected addr=lb:500515171worker2 # [ 101.825064] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:505765172worker2 # [ 101.829356] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05173worker1 # [ 102.315314] nix-grpc-daemon[768]: ts=2026-09-22T11:00:07Z level=info event=scheduler_restarting5174worker2 # [ 102.377824] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:505765175worker2 # [ 102.383005] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:07Z level=info event=worker_gone addr=192.168.1.3:500515176worker1 # [ 102.385474] nix-grpc-daemon[768]: ts=2026-09-22T11:00:07Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5177worker1 # [ 102.417187] nix-daemon[834]: reaped child process 836, status = succeeded5178worker1 # [ 103.158362] nix-grpc-daemon[768]: E0000 00:00:1790074808.753363 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5179worker1 # [ 103.175491] nix-grpc-daemon[768]: E0000 00:00:1790074808.770573 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5180worker1 # [ 104.025151] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5181worker1 # [ 104.030844] systemd[1]: Stopped Nix worker-protocol over gRPC.5182worker1 # [ 104.040491] systemd[1]: nix-grpc-daemon.service: Consumed 3.940s CPU time over 1min 25.254s wall clock time, 61.9M memory peak, 99.3K incoming IP traffic, 177.9K outgoing IP traffic.5183worker1 # [ 104.053288] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5184worker1 # [ 104.061397] systemd[1]: Closed Nix worker-protocol over gRPC.5185worker1 # [ 104.063200] systemd[1]: nix-grpc-daemon.socket: Consumed 543us CPU time over 1min 33.961s wall clock time, 256K memory peak, 92.9K incoming IP traffic, 75.8K outgoing IP traffic.5186lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5187lb: must succeed: curl -sf localhost:9901/clusters5188lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5189lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5190client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25191worker2 # [ 104.429592] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505925192worker2 # [ 104.447396] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5193worker2 # [ 104.475033] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 1/5)5194worker2 # [ 104.491622] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 2/5)5195worker2 # [ 104.537879] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 3/5)5196worker2 # [ 104.630455] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 730 ms (attempt 4/5)5197worker2 # [ 105.363758] nix-grpc-daemon[1221]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5198worker2 # [ 105.367574] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5199client # copying 2 paths...5200client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5201worker2 # [ 105.394769] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505925202client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5203worker2 # [ 105.415885] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5204worker2 # [ 105.449095] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:11Z level=info event=niks3_push_started pid=12815205worker2 # [ 105.607281] nix-grpc-daemon[1281]: time=2026-09-22T11:00:11.206Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5206worker2 # [ 105.722891] nix-daemon[855]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)5207worker2 # [ 105.742322] nix-daemon[855]: reaped child process 1297, status = succeeded5208lb # [ 105.727646] niks3-server[998]: 2026/09/22 11:00:11 INFO Received uploads request method=POST path=/api/pending_closures5209lb # [ 105.741482] niks3-server[998]: 2026/09/22 11:00:11 INFO Received uploads request method=POST path=/api/pending_closures5210worker2 # [ 105.773081] nix-grpc-daemon[1281]: time=2026-09-22T11:00:11.371Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5211worker2 # [ 105.774624] nix-grpc-daemon[1281]: time=2026-09-22T11:00:11.371Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5212worker2 # [ 105.778358] nix-grpc-daemon[1281]: time=2026-09-22T11:00:11.371Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5213lb # [ 105.785538] niks3-server[998]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5214lb # [ 105.807993] niks3-server[998]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5215lb # [ 105.821095] niks3-server[998]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5216lb # [ 105.829233] niks3-server[998]: 2026/09/22 11:00:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5217lb # [ 105.835820] niks3-server[998]: 2026/09/22 11:00:11 INFO Signed narinfos id=29 count=15218worker2 # [ 105.863429] nix-grpc-daemon[1281]: time=2026-09-22T11:00:11.460Z level=INFO msg="Uploading 3 narinfos"5219lb # [ 105.842756] niks3-server[998]: 2026/09/22 11:00:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5220lb # [ 105.848838] niks3-server[998]: 2026/09/22 11:00:11 INFO Signed narinfos id=30 count=25221lb # [ 105.853447] niks3-server[998]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5222lb # [ 105.867251] niks3-server[998]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5223lb # [ 105.877756] niks3-server[998]: 2026/09/22 11:00:11 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5224lb # [ 105.886243] niks3-server[998]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5225lb # [ 105.894006] niks3-server[998]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5226lb # [ 105.907971] niks3-server[998]: 2026/09/22 11:00:11 INFO Completed upload id=295227lb # [ 105.910672] niks3-server[998]: 2026/09/22 11:00:11 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5228lb # [ 105.914490] niks3-server[998]: 2026/09/22 11:00:11 INFO Completed upload id=305229worker2 # [ 105.936471] nix-grpc-daemon[1281]: time=2026-09-22T11:00:11.535Z level=INFO msg="Upload complete. (325ms)"5230worker2 # [ 105.948616] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50592 duration_s=0 paths=2 nar_bytes_in=3125231worker2 # [ 105.964853] nix-daemon[855]: reaped child process 1279, status = succeeded5232client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.79 seconds)5233worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5234worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5235worker1 # [ 106.053215] systemd[1]: Listening on Nix worker-protocol over gRPC.5236worker1 # [ 106.060652] systemd[1]: Starting Nix worker-protocol over gRPC...5237worker1 # [ 106.194850] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:11Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5238worker1 # [ 106.197147] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5239worker1 # [ 106.201143] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15240worker1 # [ 106.208239] nix-grpc-daemon[1226]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5241worker1 # [ 106.232548] nix-grpc-daemon[1226]: E0000 00:00:1790074811.800972 1226 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5242lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5243lb: must succeed: curl -sf localhost:9901/clusters5244worker1 # [ 106.234928] nix-grpc-daemon[1226]: E0000 00:00:1790074811.803145 1226 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5245worker1 # [ 106.258098] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5246worker1 # [ 106.260322] nix-grpc-daemon[1226]: E0000 00:00:1790074811.813747 1265 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5247worker1 # [ 106.262818] nix-grpc-daemon[1226]: E0000 00:00:1790074811.836502 1265 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5248worker1 # [ 106.265089] nix-grpc-daemon[1226]: E0000 00:00:1790074811.836674 1265 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5249worker1 # [ 106.267342] nix-grpc-daemon[1226]: E0000 00:00:1790074811.852189 1265 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5250worker1 # [ 106.269720] systemd[1]: Started Nix worker-protocol over gRPC.5251worker1 # [ 106.277829] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:11Z level=info event=scheduler_connected addr=lb:500515252worker2 # [ 106.322254] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:11Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:505765253worker2 # [ 106.327765] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05254lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5255lb: must succeed: curl -sf localhost:9901/clusters5256lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5257lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5258(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.46 seconds)5259subtest: a niks3 restart does not cost the leader its role5260lb: must succeed: curl -sf localhost:9901/clusters5261lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5262lb: must succeed: curl -sf localhost:9901/clusters5263lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5264worker2 # [ 107.562500] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:13Z level=info event=scheduler_lock_lost status=2005265worker1 # [ 107.519709] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:13Z level=info event=scheduler_lock_lost status=2005266lb # [ 107.540589] niks3-server[998]: 2026/09/22 11:00:13 INFO Shutdown signal received, draining in-flight requests timeout=10s5267lb # [ 107.540778] niks3-server[998]: 2026/09/22 11:00:13 INFO lead: released remote=192.168.1.4:538145268lb # [ 107.552681] systemd[1]: Stopping niks3 server...5269lb # [ 107.568275] systemd[1]: niks3.service: Deactivated successfully.5270lb # [ 107.570718] systemd[1]: Stopped niks3 server.5271lb # [ 107.571471] systemd[1]: niks3.service: Consumed 569ms CPU time over 1min 20.537s wall clock time, 26.2M memory peak, 77.9K incoming IP traffic, 112.8K outgoing IP traffic.5272lb # [ 107.581207] systemd[1]: Starting niks3 server...5273lb # [ 107.637013] niks3-server[1138]: 2026/09/22 11:00:13 goose: no migrations to run. current version: 202609200000005274lb # [ 107.643464] niks3-server[1138]: 2026/09/22 11:00:13 OK 1_commit_pending_closure.sql (4.64ms)5275lb # [ 107.647059] niks3-server[1138]: 2026/09/22 11:00:13 OK 2_object_stats_trigger.sql (3.23ms)5276lb # [ 107.648991] niks3-server[1138]: 2026/09/22 11:00:13 goose: up to current file version: 25277lb # [ 107.650641] niks3-server[1138]: 2026/09/22 11:00:13 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5278lb # [ 107.661402] niks3-server[1138]: 2026/09/22 11:00:13 INFO Read proxy enabled — serving cache objects from S35279lb # [ 107.662717] niks3-server[1138]: 2026/09/22 11:00:13 INFO Using socket-activated listener address=0.0.0.0:57515280lb # [ 107.667411] niks3-server[1138]: 2026/09/22 11:00:13 INFO systemd watchdog enabled interval=15s5281lb # [ 107.671240] niks3-server[1138]: 2026/09/22 11:00:13 INFO Starting HTTP server address=0.0.0.0:57515282lb: waiting for unit niks3.service5283lb # [ 107.673785] systemd[1]: Started niks3 server.5284lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5285lb: must succeed: curl -sf localhost:9901/clusters5286lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5287worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5288worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5289lb: must succeed: curl -sf localhost:9901/clusters5290lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5291lb: must succeed: curl -sf localhost:9901/clusters5292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5293worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5294worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5295client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25296worker1 # [ 108.127218] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470145297worker1 # [ 108.142116] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5298client # don't know how to build these paths:5299client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5300worker1 # [ 108.165686] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470145301worker1 # [ 108.180269] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)5302worker1 # [ 108.245444] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 2/5)5303worker1 # [ 108.322359] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 3/5)5304lb # [ 108.546280] niks3-server[1138]: 2026/09/22 11:00:14 INFO lead: acquired remote=192.168.1.4:538705305worker1 # [ 108.599414] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 4/5)5306worker1 # [ 108.692245] nix-grpc-daemon[1226]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5307worker1 # [ 108.698900] nix-grpc-daemon[1226]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5308worker2 # [ 108.790009] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:14Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05309worker2 # [ 108.790428] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:14Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15310lb # [ 108.790493] envoy[898]: conn peer=[2001:db8:1::4]:37328 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=73016 tx=3791 ms=803655311lb # [ 108.795811] envoy[898]: conn peer=[2001:db8:1::1]:39052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=295312lb # [ 108.800752] envoy[898]: conn peer=[2001:db8:1::3]:47196 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=16215 tx=4346 ms=832735313lb # [ 108.805748] envoy[898]: conn peer=[2001:db8:1::1]:58986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1007 tx=305 ms=15395314worker2 # [ 108.836824] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505925315worker2 # [ 108.856380] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505925316client # copying 1 paths to 'lb:50051'...5317client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5318worker2 # [ 108.870756] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505925319worker2 # [ 108.886401] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5320worker2 # [ 109.040841] nix-daemon[855]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5321worker2 # [ 109.064605] nix-daemon[855]: reaped child process 1319, status = succeeded5322worker2 # [ 109.161569] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5323worker2 # [ 109.177372] nix-daemon[855]: reaped child process 1324, status = succeeded5324lb # [ 109.163842] niks3-server[1138]: 2026/09/22 11:00:14 INFO Received uploads request method=POST path=/api/pending_closures5325worker2 # [ 109.200302] nix-grpc-daemon[1281]: time=2026-09-22T11:00:14.798Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5326worker2 # [ 109.201878] nix-grpc-daemon[1281]: time=2026-09-22T11:00:14.799Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5327lb # [ 109.211009] niks3-server[1138]: 2026/09/22 11:00:14 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5328lb # [ 109.225922] niks3-server[1138]: 2026/09/22 11:00:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5329worker2 # [ 109.254896] nix-grpc-daemon[1281]: time=2026-09-22T11:00:14.853Z level=INFO msg="Uploading 1 narinfos"5330lb # [ 109.236623] niks3-server[1138]: 2026/09/22 11:00:14 INFO Signed narinfos id=31 count=15331lb # [ 109.246114] niks3-server[1138]: 2026/09/22 11:00:14 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5332lb # [ 109.253818] niks3-server[1138]: 2026/09/22 11:00:14 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5333lb # [ 109.271641] niks3-server[1138]: 2026/09/22 11:00:14 INFO Completed upload id=315334worker2 # [ 109.293574] nix-grpc-daemon[1281]: time=2026-09-22T11:00:14.892Z level=INFO msg="Upload complete. (389ms)"5335worker2 # [ 109.304345] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50592 duration_s=0 paths=1 nar_bytes_in=5445336worker2 # [ 109.316569] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505925337worker2 # [ 109.328784] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5338worker2 # [ 109.339173] nix-daemon[855]: reaped child process 1314, status = succeeded5339client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5340client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5341worker2 # [ 109.346897] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5342client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5343client # farm-a> LOG-farm-a-after-niks3-restart5344worker2 # [ 110.244600] nix-daemon[855]: reaped child process 1328, status = succeeded5345worker2 # [ 110.316657] nix-daemon[855]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)5346worker2 # [ 110.336110] nix-daemon[855]: reaped child process 1347, status = succeeded5347lb # [ 110.321994] niks3-server[1138]: 2026/09/22 11:00:15 INFO Received uploads request method=POST path=/api/pending_closures5348worker2 # [ 110.356476] nix-grpc-daemon[1281]: time=2026-09-22T11:00:15.955Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5349worker2 # [ 110.358245] nix-grpc-daemon[1281]: time=2026-09-22T11:00:15.955Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5350lb # [ 110.377911] niks3-server[1138]: 2026/09/22 11:00:15 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5351lb # [ 110.388852] niks3-server[1138]: 2026/09/22 11:00:15 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5352lb # [ 110.399668] niks3-server[1138]: 2026/09/22 11:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5353worker2 # [ 110.422367] nix-grpc-daemon[1281]: time=2026-09-22T11:00:16.020Z level=INFO msg="Uploading 1 narinfos"5354lb # [ 110.406092] niks3-server[1138]: 2026/09/22 11:00:16 INFO Signed narinfos id=32 count=15355lb # [ 110.410640] niks3-server[1138]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5356lb # [ 110.418288] niks3-server[1138]: 2026/09/22 11:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5357lb # [ 110.424143] niks3-server[1138]: 2026/09/22 11:00:16 INFO Completed upload id=325358worker2 # [ 110.445201] nix-grpc-daemon[1281]: time=2026-09-22T11:00:16.044Z level=INFO msg="Upload complete. (288ms)"5359lb # [ 110.427474] niks3-server[1138]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5360worker2 # [ 110.469262] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50592 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15361worker2 # [ 110.484321] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:16Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05362worker2 # [ 110.489970] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:16Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15363worker2 # [ 110.495220] nix-daemon[855]: reaped child process 1326, status = succeeded5364worker2 # [ 110.510915] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505925365worker2 # [ 110.525966] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505925366worker2 # [ 110.541162] nix-grpc-daemon[1281]: time=2026-09-22T11:00:16.140Z level=INFO msg="All 1 paths already cached"5367client # copying 1 paths to 'lb:50051'...5368client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5369worker2 # [ 110.551262] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505925370worker2 # [ 110.566128] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5371worker2 # [ 110.716569] nix-daemon[855]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)5372worker2 # [ 110.734794] nix-daemon[855]: reaped child process 1356, status = succeeded5373worker2 # [ 110.837347] nix-daemon[855]: accepted connection from pid 1358, user nix-grpc-daemon (trusted)5374worker2 # [ 110.856775] nix-daemon[855]: reaped child process 1361, status = succeeded5375lb # [ 110.833113] niks3-server[1138]: 2026/09/22 11:00:16 INFO Received uploads request method=POST path=/api/pending_closures5376worker2 # [ 110.865156] nix-grpc-daemon[1281]: time=2026-09-22T11:00:16.464Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5377worker2 # [ 110.866648] nix-grpc-daemon[1281]: time=2026-09-22T11:00:16.464Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5378lb # [ 110.864200] niks3-server[1138]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5379worker2 # [ 110.897273] nix-grpc-daemon[1281]: time=2026-09-22T11:00:16.496Z level=INFO msg="Uploading 1 narinfos"5380lb # [ 110.880469] niks3-server[1138]: 2026/09/22 11:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5381lb # [ 110.890789] niks3-server[1138]: 2026/09/22 11:00:16 INFO Signed narinfos id=33 count=15382lb # [ 110.896148] niks3-server[1138]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5383lb # [ 110.905562] niks3-server[1138]: 2026/09/22 11:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5384worker2 # [ 110.926634] nix-grpc-daemon[1281]: time=2026-09-22T11:00:16.525Z level=INFO msg="Upload complete. (344ms)"5385lb # [ 110.911273] niks3-server[1138]: 2026/09/22 11:00:16 INFO Completed upload id=335386worker2 # [ 110.934571] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50592 duration_s=0 paths=1 nar_bytes_in=6565387lb # [ 110.914481] niks3-server[1138]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5388worker2 # [ 110.946147] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505925389worker2 # [ 110.951763] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5390worker2 # [ 110.956212] nix-daemon[855]: reaped child process 1351, status = succeeded5391client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5392worker2 # [ 110.968990] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5393client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5394client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5395client # farm-b> LOG-farm-b-after-niks3-restart5396worker2 # [ 112.452876] nix-daemon[855]: reaped child process 1365, status = succeeded5397worker2 # [ 112.548577] nix-daemon[855]: accepted connection from pid 1381, user nix-grpc-daemon (trusted)5398worker2 # [ 112.569498] nix-daemon[855]: reaped child process 1384, status = succeeded5399lb # [ 112.554597] niks3-server[1138]: 2026/09/22 11:00:18 INFO Received uploads request method=POST path=/api/pending_closures5400worker2 # [ 112.584774] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.183Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5401worker2 # [ 112.586474] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.183Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5402lb # [ 112.595640] niks3-server[1138]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5403lb # [ 112.616189] niks3-server[1138]: 2026/09/22 11:00:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5404worker2 # [ 112.647064] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.245Z level=INFO msg="Uploading 1 narinfos"5405lb # [ 112.630672] niks3-server[1138]: 2026/09/22 11:00:18 INFO Signed narinfos id=34 count=15406lb # [ 112.645933] niks3-server[1138]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5407lb # [ 112.655918] niks3-server[1138]: 2026/09/22 11:00:18 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5408worker2 # [ 112.682746] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.282Z level=INFO msg="Upload complete. (303ms)"5409lb # [ 112.664527] niks3-server[1138]: 2026/09/22 11:00:18 INFO Completed upload id=345410lb # [ 112.671028] niks3-server[1138]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5411lb # [ 112.677515] niks3-server[1138]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5412worker2 # [ 112.698639] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50592 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15413worker2 # [ 112.722744] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:18Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05414worker2 # [ 112.727865] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:18Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15415worker2 # [ 112.733665] nix-daemon[855]: reaped child process 1363, status = succeeded5416worker2 # [ 112.758184] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505925417worker2 # [ 112.773496] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505925418worker2 # [ 112.789539] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.388Z level=INFO msg="All 2 paths already cached"5419client # copying 1 paths to 'lb:50051'...5420client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5421worker2 # [ 112.799729] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505925422worker2 # [ 112.814637] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5423worker2 # [ 112.968144] nix-daemon[855]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5424worker2 # [ 112.987713] nix-daemon[855]: reaped child process 1393, status = succeeded5425worker2 # [ 113.089057] nix-daemon[855]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5426worker2 # [ 113.107407] nix-daemon[855]: reaped child process 1398, status = succeeded5427lb # [ 113.089442] niks3-server[1138]: 2026/09/22 11:00:18 INFO Received uploads request method=POST path=/api/pending_closures5428worker2 # [ 113.121174] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.720Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5429worker2 # [ 113.122693] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.720Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5430lb # [ 113.122190] niks3-server[1138]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5431lb # [ 113.137188] niks3-server[1138]: 2026/09/22 11:00:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5432worker2 # [ 113.166810] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.765Z level=INFO msg="Uploading 1 narinfos"5433lb # [ 113.148939] niks3-server[1138]: 2026/09/22 11:00:18 INFO Signed narinfos id=35 count=15434lb # [ 113.151665] niks3-server[1138]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5435lb # [ 113.164609] niks3-server[1138]: 2026/09/22 11:00:18 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5436lb # [ 113.179450] niks3-server[1138]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5437lb # [ 113.183939] niks3-server[1138]: 2026/09/22 11:00:18 INFO Completed upload id=355438worker2 # [ 113.205785] nix-grpc-daemon[1281]: time=2026-09-22T11:00:18.805Z level=INFO msg="Upload complete. (374ms)"5439worker2 # [ 113.214636] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50592 duration_s=0 paths=1 nar_bytes_in=7845440worker2 # [ 113.231028] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505925441worker2 # [ 113.238362] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5442worker2 # [ 113.243810] nix-daemon[855]: reaped child process 1388, status = succeeded5443client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5444worker2 # [ 113.256825] nix-daemon[855]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5445client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5446client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5447client # farm-top> LOG-farm-top-after-niks3-restart5448worker2 # [ 114.503814] nix-daemon[855]: reaped child process 1402, status = succeeded5449worker2 # [ 114.620253] nix-daemon[855]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5450worker2 # [ 114.638452] nix-daemon[855]: reaped child process 1421, status = succeeded5451lb # [ 114.622916] niks3-server[1138]: 2026/09/22 11:00:20 INFO Received uploads request method=POST path=/api/pending_closures5452worker2 # [ 114.656473] nix-grpc-daemon[1281]: time=2026-09-22T11:00:20.254Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5453worker2 # [ 114.658257] nix-grpc-daemon[1281]: time=2026-09-22T11:00:20.254Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5454lb # [ 114.652492] niks3-server[1138]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5455lb # [ 114.676065] niks3-server[1138]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5456worker2 # [ 114.719590] nix-grpc-daemon[1281]: time=2026-09-22T11:00:20.318Z level=INFO msg="Uploading 1 narinfos"5457lb # [ 114.690107] niks3-server[1138]: 2026/09/22 11:00:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5458lb # [ 114.703782] niks3-server[1138]: 2026/09/22 11:00:20 INFO Signed narinfos id=36 count=15459lb # [ 114.708766] niks3-server[1138]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5460lb # [ 114.719357] niks3-server[1138]: 2026/09/22 11:00:20 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5461worker2 # [ 114.748449] nix-grpc-daemon[1281]: time=2026-09-22T11:00:20.347Z level=INFO msg="Upload complete. (279ms)"5462lb # [ 114.728840] niks3-server[1138]: 2026/09/22 11:00:20 INFO Completed upload id=365463lb # [ 114.731050] niks3-server[1138]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5464worker2 # [ 114.761183] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50592 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15465worker2 # [ 114.781132] nix-daemon[855]: reaped child process 1400, status = succeeded5466client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.86 seconds)5467client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5468client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5469(finished: subtest: a niks3 restart does not cost the leader its role, in 7.54 seconds)5470subtest: scheduler down: the other node takes the lock and keeps it5471lb: must succeed: curl -sf localhost:9901/clusters5472lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.13 seconds)5473lb: must succeed: curl -sf localhost:9901/clusters5474lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5475lb: must succeed: curl -sf localhost:9901/clusters5476lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5477lb: must succeed: curl -sf localhost:9901/clusters5478lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5479worker2 # [ 115.279465] systemd[1]: Stopping Nix worker-protocol over gRPC...5480worker2 # [ 115.654342] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:21Z level=info event=signal_exit5481lb # [ 116.553394] niks3-server[1138]: 2026/09/22 11:00:22 INFO lead: released remote=192.168.1.4:538705482worker1 # [ 117.533237] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_restarting addr=lb:500515483worker2 # [ 117.579810] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:23Z level=info event=scheduler_restarting5484worker2 # [ 117.584454] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:23Z level=info event=scheduler_restarting addr=lb:500515485worker2 # [ 117.588989] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:505765486worker2 # [ 117.594013] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:23Z level=info event=worker_gone addr=192.168.1.4:500515487worker2 # [ 117.598713] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:505765488worker2 # [ 117.603664] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:23Z level=info event=worker_gone addr=192.168.1.3:500515489worker2 # [ 117.628685] nix-daemon[855]: reaped child process 1273, status = succeeded5490worker2 # [ 117.723103] nix-grpc-daemon[1221]: E0000 00:00:1790074823.322040 1266 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5491worker2 # [ 117.729766] nix-grpc-daemon[1221]: E0000 00:00:1790074823.322350 1266 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5492worker1 # [ 117.732961] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_connected addr=lb:500515493worker1 # [ 117.738553] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5494worker2 # [ 117.794835] nix-grpc-daemon[1221]: ts=2026-09-22T11:00:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5495worker2 # [ 117.851027] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5496worker2 # [ 117.858206] systemd[1]: Stopped Nix worker-protocol over gRPC.5497worker2 # [ 117.866551] systemd[1]: nix-grpc-daemon.service: Consumed 1.713s CPU time over 16.415s wall clock time, 38.3M memory peak, 45.6K incoming IP traffic, 76.1K outgoing IP traffic.5498worker2 # [ 117.880248] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5499worker2 # [ 117.882207] systemd[1]: Closed Nix worker-protocol over gRPC.5500worker2 # [ 117.883834] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 17.292s wall clock time, 256K memory peak, 38.7K incoming IP traffic, 28.9K outgoing IP traffic.5501worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5502worker1 # [ 117.939243] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_connected addr=lb:500515503worker1 # [ 117.944325] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5504worker1 # [ 118.144947] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_connected addr=lb:500515505worker1 # [ 118.150205] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5506worker1 # [ 118.351008] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_connected addr=lb:500515507worker1 # [ 118.356454] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5508worker1 # [ 118.527704] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=scheduler_take_over5509lb # [ 118.549470] niks3-server[1138]: 2026/09/22 11:00:24 INFO lead: acquired remote=192.168.1.3:360745510worker1 # [ 118.558951] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=scheduler_connected addr=lb:500515511worker1 # [ 118.563189] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5512worker1 # [ 118.759815] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=scheduler_connected addr=lb:500515513worker1 # [ 118.765126] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5514lb # [ 118.789863] envoy[898]: conn peer=[2001:db8:1::1]:59002 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9753 tx=4129 ms=66115515lb # [ 118.795719] envoy[898]: conn peer=[2001:db8:1::4]:58194 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1665 tx=425 ms=160655516worker1 # [ 118.965945] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=scheduler_connected addr=lb:500515517worker1 # [ 118.977324] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5518worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.17 seconds)5519worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5520worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5521worker1 # [ 119.173720] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=scheduler_connected addr=lb:500515522worker1 # [ 119.183260] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:386665523worker1 # [ 119.188873] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05524worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5525worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5526client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25527client # don't know how to build these paths:5528client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5529worker1 # [ 120.349943] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470145530worker1 # [ 120.357529] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470145531worker1 # [ 120.382250] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:25Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05532worker1 # [ 120.389494] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:25Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15533worker1 # [ 120.426827] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470145534worker1 # [ 120.442507] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470145535client # copying 1 paths to 'lb:50051'...5536client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5537worker1 # [ 120.461349] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470145538worker1 # [ 120.476542] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5539worker1 # [ 120.505325] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:26Z level=info event=niks3_push_started pid=13085540worker1 # [ 120.662651] nix-grpc-daemon[1308]: time=2026-09-22T11:00:26.257Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5541worker1 # [ 120.775012] nix-daemon[834]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5542worker1 # [ 120.793147] nix-daemon[834]: reaped child process 1324, status = succeeded5543worker1 # [ 120.898495] nix-daemon[834]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5544worker1 # [ 120.916372] nix-daemon[834]: reaped child process 1329, status = succeeded5545lb # [ 120.945015] niks3-server[1138]: 2026/09/22 11:00:26 INFO Received uploads request method=POST path=/api/pending_closures5546worker1 # [ 120.934655] nix-grpc-daemon[1308]: time=2026-09-22T11:00:26.528Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5547worker1 # [ 120.936525] nix-grpc-daemon[1308]: time=2026-09-22T11:00:26.528Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5548lb # [ 120.980552] niks3-server[1138]: 2026/09/22 11:00:26 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5549worker1 # [ 120.984636] nix-grpc-daemon[1308]: time=2026-09-22T11:00:26.579Z level=INFO msg="Uploading 1 narinfos"5550lb # [ 121.006461] niks3-server[1138]: 2026/09/22 11:00:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5551lb # [ 121.015722] niks3-server[1138]: 2026/09/22 11:00:26 INFO Signed narinfos id=37 count=15552lb # [ 121.028692] niks3-server[1138]: 2026/09/22 11:00:26 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5553worker1 # [ 121.014254] nix-grpc-daemon[1308]: time=2026-09-22T11:00:26.609Z level=INFO msg="Upload complete. (347ms)"5554lb # [ 121.040314] niks3-server[1138]: 2026/09/22 11:00:26 INFO Completed upload id=375555worker1 # [ 121.025223] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47014 duration_s=0 paths=1 nar_bytes_in=5205556worker1 # [ 121.038571] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470145557worker1 # [ 121.045071] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5558worker1 # [ 121.050207] nix-daemon[834]: reaped child process 1306, status = succeeded5559worker1 # [ 121.060179] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5560client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5561client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5562client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5563client # farm-a> LOG-farm-a-on-standby5564worker1 # [ 121.960705] nix-daemon[834]: reaped child process 1333, status = succeeded5565worker1 # [ 122.039684] nix-daemon[834]: accepted connection from pid 1349, user nix-grpc-daemon (trusted)5566worker1 # [ 122.057488] nix-daemon[834]: reaped child process 1352, status = succeeded5567lb # [ 122.085395] niks3-server[1138]: 2026/09/22 11:00:27 INFO Received uploads request method=POST path=/api/pending_closures5568worker1 # [ 122.080477] nix-grpc-daemon[1308]: time=2026-09-22T11:00:27.674Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5569worker1 # [ 122.082427] nix-grpc-daemon[1308]: time=2026-09-22T11:00:27.674Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5570lb # [ 122.147663] niks3-server[1138]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5571lb # [ 122.157561] niks3-server[1138]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5572lb # [ 122.168810] niks3-server[1138]: 2026/09/22 11:00:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5573worker1 # [ 122.148117] nix-grpc-daemon[1308]: time=2026-09-22T11:00:27.741Z level=INFO msg="Uploading 1 narinfos"5574lb # [ 122.171780] niks3-server[1138]: 2026/09/22 11:00:27 INFO Signed narinfos id=38 count=15575lb # [ 122.181195] niks3-server[1138]: 2026/09/22 11:00:27 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5576lb # [ 122.190484] niks3-server[1138]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5577worker1 # [ 122.168395] nix-grpc-daemon[1308]: time=2026-09-22T11:00:27.763Z level=INFO msg="Upload complete. (297ms)"5578lb # [ 122.199833] niks3-server[1138]: 2026/09/22 11:00:27 INFO Completed upload id=385579lb # [ 122.202721] niks3-server[1138]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5580worker1 # [ 122.192951] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47014 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15581worker1 # [ 122.207056] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:27Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05582worker1 # [ 122.211301] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:27Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15583worker1 # [ 122.215605] nix-daemon[834]: reaped child process 1331, status = succeeded5584worker1 # [ 122.235388] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470145585worker1 # [ 122.250850] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470145586worker1 # [ 122.267088] nix-grpc-daemon[1308]: time=2026-09-22T11:00:27.862Z level=INFO msg="All 1 paths already cached"5587client # copying 1 paths to 'lb:50051'...5588client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5589worker1 # [ 122.277333] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470145590worker1 # [ 122.290889] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5591worker1 # [ 122.441469] nix-daemon[834]: accepted connection from pid 1358, user nix-grpc-daemon (trusted)5592worker1 # [ 122.461238] nix-daemon[834]: reaped child process 1361, status = succeeded5593worker1 # [ 122.567077] nix-daemon[834]: accepted connection from pid 1363, user nix-grpc-daemon (trusted)5594worker1 # [ 122.585198] nix-daemon[834]: reaped child process 1366, status = succeeded5595lb # [ 122.609808] niks3-server[1138]: 2026/09/22 11:00:28 INFO Received uploads request method=POST path=/api/pending_closures5596worker1 # [ 122.601957] nix-grpc-daemon[1308]: time=2026-09-22T11:00:28.196Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5597worker1 # [ 122.603461] nix-grpc-daemon[1308]: time=2026-09-22T11:00:28.196Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5598lb # [ 122.643930] niks3-server[1138]: 2026/09/22 11:00:28 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5599worker1 # [ 122.636153] nix-grpc-daemon[1308]: time=2026-09-22T11:00:28.231Z level=INFO msg="Uploading 1 narinfos"5600lb # [ 122.660244] niks3-server[1138]: 2026/09/22 11:00:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5601lb # [ 122.673864] niks3-server[1138]: 2026/09/22 11:00:28 INFO Signed narinfos id=39 count=15602lb # [ 122.682465] niks3-server[1138]: 2026/09/22 11:00:28 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5603lb # [ 122.695041] niks3-server[1138]: 2026/09/22 11:00:28 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5604lb # [ 122.700770] niks3-server[1138]: 2026/09/22 11:00:28 INFO Completed upload id=395605worker1 # [ 122.678008] nix-grpc-daemon[1308]: time=2026-09-22T11:00:28.273Z level=INFO msg="Upload complete. (370ms)"5606lb # [ 122.702841] niks3-server[1138]: 2026/09/22 11:00:28 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5607worker1 # [ 122.688268] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47014 duration_s=0 paths=1 nar_bytes_in=6405608worker1 # [ 122.703454] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470145609worker1 # [ 122.713323] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5610worker1 # [ 122.716382] nix-daemon[834]: reaped child process 1356, status = succeeded5611client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5612client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5613worker1 # [ 122.732421] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5614client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5615client # farm-b> LOG-farm-b-on-standby5616worker1 # [ 123.984361] nix-daemon[834]: reaped child process 1370, status = succeeded5617worker1 # [ 124.106328] nix-daemon[834]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)5618worker1 # [ 124.123284] nix-daemon[834]: reaped child process 1389, status = succeeded5619lb # [ 124.151659] niks3-server[1138]: 2026/09/22 11:00:29 INFO Received uploads request method=POST path=/api/pending_closures5620worker1 # [ 124.141765] nix-grpc-daemon[1308]: time=2026-09-22T11:00:29.736Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5621worker1 # [ 124.143306] nix-grpc-daemon[1308]: time=2026-09-22T11:00:29.736Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5622lb # [ 124.185601] niks3-server[1138]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5623worker1 # [ 124.181905] nix-grpc-daemon[1308]: time=2026-09-22T11:00:29.776Z level=INFO msg="Uploading 1 narinfos"5624lb # [ 124.203132] niks3-server[1138]: 2026/09/22 11:00:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5625lb # [ 124.215016] niks3-server[1138]: 2026/09/22 11:00:29 INFO Signed narinfos id=40 count=15626lb # [ 124.221282] niks3-server[1138]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5627lb # [ 124.227944] niks3-server[1138]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5628lb # [ 124.236996] niks3-server[1138]: 2026/09/22 11:00:29 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5629worker1 # [ 124.216113] nix-grpc-daemon[1308]: time=2026-09-22T11:00:29.811Z level=INFO msg="Upload complete. (272ms)"5630lb # [ 124.239655] niks3-server[1138]: 2026/09/22 11:00:29 INFO Completed upload id=405631lb # [ 124.243611] niks3-server[1138]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5632worker1 # [ 124.235104] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47014 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15633worker1 # [ 124.251764] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:29Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05634worker1 # [ 124.255829] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:29Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15635worker1 # [ 124.260191] nix-daemon[834]: reaped child process 1368, status = succeeded5636worker1 # [ 124.279298] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470145637worker1 # [ 124.294385] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470145638worker1 # [ 124.308640] nix-grpc-daemon[1308]: time=2026-09-22T11:00:29.903Z level=INFO msg="All 2 paths already cached"5639client # copying 1 paths to 'lb:50051'...5640client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5641worker1 # [ 124.319052] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470145642worker1 # [ 124.333006] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5643worker1 # [ 124.487862] nix-daemon[834]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5644worker1 # [ 124.511689] nix-daemon[834]: reaped child process 1398, status = succeeded5645worker1 # [ 124.608115] nix-daemon[834]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5646worker1 # [ 124.628923] nix-daemon[834]: reaped child process 1403, status = succeeded5647lb # [ 124.651143] niks3-server[1138]: 2026/09/22 11:00:30 INFO Received uploads request method=POST path=/api/pending_closures5648worker1 # [ 124.639757] nix-grpc-daemon[1308]: time=2026-09-22T11:00:30.234Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5649worker1 # [ 124.641417] nix-grpc-daemon[1308]: time=2026-09-22T11:00:30.234Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5650lb # [ 124.679241] niks3-server[1138]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5651lb # [ 124.694913] niks3-server[1138]: 2026/09/22 11:00:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5652worker1 # [ 124.679661] nix-grpc-daemon[1308]: time=2026-09-22T11:00:30.274Z level=INFO msg="Uploading 1 narinfos"5653lb # [ 124.707798] niks3-server[1138]: 2026/09/22 11:00:30 INFO Signed narinfos id=41 count=15654lb # [ 124.713467] niks3-server[1138]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5655worker1 # [ 124.702519] nix-grpc-daemon[1308]: time=2026-09-22T11:00:30.297Z level=INFO msg="Upload complete. (353ms)"5656lb # [ 124.724779] niks3-server[1138]: 2026/09/22 11:00:30 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5657lb # [ 124.733735] niks3-server[1138]: 2026/09/22 11:00:30 INFO Completed upload id=415658worker1 # [ 124.713004] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47014 duration_s=0 paths=1 nar_bytes_in=7685659lb # [ 124.736345] niks3-server[1138]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5660worker1 # [ 124.722488] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470145661worker1 # [ 124.728794] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5662worker1 # [ 124.733479] nix-daemon[834]: reaped child process 1393, status = succeeded5663client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5664worker1 # [ 124.746354] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5665client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5666client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5667client # farm-top> LOG-farm-top-on-standby5668worker1 # [ 125.655778] nix-daemon[834]: reaped child process 1407, status = succeeded5669worker1 # [ 125.725155] nix-daemon[834]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5670worker1 # [ 125.747802] nix-daemon[834]: reaped child process 1426, status = succeeded5671lb # [ 125.770025] niks3-server[1138]: 2026/09/22 11:00:31 INFO Received uploads request method=POST path=/api/pending_closures5672worker1 # [ 125.758557] nix-grpc-daemon[1308]: time=2026-09-22T11:00:31.352Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5673worker1 # [ 125.760148] nix-grpc-daemon[1308]: time=2026-09-22T11:00:31.352Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5674lb # [ 125.814435] niks3-server[1138]: 2026/09/22 11:00:31 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5675lb # [ 125.823514] niks3-server[1138]: 2026/09/22 11:00:31 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5676worker1 # [ 125.810780] nix-grpc-daemon[1308]: time=2026-09-22T11:00:31.405Z level=INFO msg="Uploading 1 narinfos"5677lb # [ 125.834064] niks3-server[1138]: 2026/09/22 11:00:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5678lb # [ 125.843451] niks3-server[1138]: 2026/09/22 11:00:31 INFO Signed narinfos id=42 count=15679lb # [ 125.852887] niks3-server[1138]: 2026/09/22 11:00:31 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5680worker1 # [ 125.837584] nix-grpc-daemon[1308]: time=2026-09-22T11:00:31.432Z level=INFO msg="Upload complete. (280ms)"5681lb # [ 125.862355] niks3-server[1138]: 2026/09/22 11:00:31 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5682lb # [ 125.867828] niks3-server[1138]: 2026/09/22 11:00:31 INFO Completed upload id=425683lb # [ 125.871294] niks3-server[1138]: 2026/09/22 11:00:31 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5684worker1 # [ 125.849210] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47014 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15685worker1 # [ 125.869852] nix-daemon[834]: reaped child process 1405, status = succeeded5686client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.78 seconds)5687client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5688client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5689worker2 # [ 126.203184] systemd[1]: Listening on Nix worker-protocol over gRPC.5690worker2 # [ 126.212427] systemd[1]: Starting scheduler-token.service...5691worker2 # [ 126.286001] systemd[1]: scheduler-token.service: Deactivated successfully.5692worker2 # [ 126.292182] systemd[1]: Finished scheduler-token.service.5693worker2 # [ 126.293743] systemd[1]: scheduler-token.service: Consumed 43ms CPU time over 77ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5694worker2 # [ 126.301569] systemd[1]: Starting Nix worker-protocol over gRPC...5695worker2 # [ 126.426431] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5696worker2 # [ 126.428729] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5697worker2 # [ 126.432544] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15698worker2 # [ 126.439080] nix-grpc-daemon[1442]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5699worker2 # [ 126.456191] nix-grpc-daemon[1442]: E0000 00:00:1790074832.036152 1442 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5700worker2 # [ 126.458563] nix-grpc-daemon[1442]: E0000 00:00:1790074832.038246 1442 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5701lb: must succeed: curl -sf localhost:9901/clusters5702worker2 # [ 126.487173] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5703worker2 # [ 126.489493] nix-grpc-daemon[1442]: E0000 00:00:1790074832.046325 1481 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5704worker2 # [ 126.491950] nix-grpc-daemon[1442]: E0000 00:00:1790074832.069582 1481 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5705worker2 # [ 126.494421] nix-grpc-daemon[1442]: E0000 00:00:1790074832.069741 1481 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5706worker2 # [ 126.496783] nix-grpc-daemon[1442]: E0000 00:00:1790074832.085574 1481 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5707worker2 # [ 126.502178] systemd[1]: Started Nix worker-protocol over gRPC.5708worker2 # [ 126.504766] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:32Z level=info event=scheduler_connected addr=lb:500515709worker1 # [ 126.472195] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=info event=oidc_jwks provider=mock keys=15710worker1 # [ 126.477376] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:386665711worker1 # [ 126.482792] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05712lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5713worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5714worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5715lb: must succeed: curl -sf localhost:9901/clusters5716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5717lb: must succeed: curl -sf localhost:9901/clusters5718lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5719worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5720worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5721(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.81 seconds)5722subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5723client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5724client # [ 126.818862] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5725client # Running as unit: rs.service; invocation ID: 10b6ad9f9e2e468b99d5e8565db7938f5726client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5727client # [ 127.028354] nix[1339]: don't know how to build these paths:5728client # [ 127.029434] nix[1339]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5729worker1 # [ 127.028999] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470145730worker1 # [ 127.042475] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05731worker1 # [ 127.049778] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15732worker1 # [ 127.087234] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470145733worker1 # [ 127.102944] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470145734client # [ 127.112845] nix[1339]: copying 1 paths to 'lb:50051'...5735client # [ 127.117444] nix[1339]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5736worker1 # [ 127.120493] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470145737worker1 # [ 127.135815] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5738worker1 # [ 127.295498] nix-daemon[834]: accepted connection from pid 1439, user nix-grpc-daemon (trusted)5739worker1 # [ 127.317516] nix-daemon[834]: reaped child process 1442, status = succeeded5740worker1 # [ 127.421521] nix-daemon[834]: accepted connection from pid 1444, user nix-grpc-daemon (trusted)5741worker1 # [ 127.441141] nix-daemon[834]: reaped child process 1447, status = succeeded5742lb # [ 127.466908] niks3-server[1138]: 2026/09/22 11:00:33 INFO Received uploads request method=POST path=/api/pending_closures5743worker1 # [ 127.457632] nix-grpc-daemon[1308]: time=2026-09-22T11:00:33.052Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5744worker1 # [ 127.459148] nix-grpc-daemon[1308]: time=2026-09-22T11:00:33.052Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5745lb # [ 127.495661] niks3-server[1138]: 2026/09/22 11:00:33 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5746lb # [ 127.509562] niks3-server[1138]: 2026/09/22 11:00:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5747worker1 # [ 127.494011] nix-grpc-daemon[1308]: time=2026-09-22T11:00:33.089Z level=INFO msg="Uploading 1 narinfos"5748lb # [ 127.519047] niks3-server[1138]: 2026/09/22 11:00:33 INFO Signed narinfos id=43 count=15749lb # [ 127.525303] niks3-server[1138]: 2026/09/22 11:00:33 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5750lb # [ 127.535485] niks3-server[1138]: 2026/09/22 11:00:33 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5751lb # [ 127.541973] niks3-server[1138]: 2026/09/22 11:00:33 INFO Completed upload id=435752worker1 # [ 127.518134] nix-grpc-daemon[1308]: time=2026-09-22T11:00:33.113Z level=INFO msg="Upload complete. (365ms)"5753lb # [ 127.547776] niks3-server[1138]: 2026/09/22 11:00:33 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5754worker1 # [ 127.524377] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47014 duration_s=0 paths=1 nar_bytes_in=4165755worker1 # [ 127.538914] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470145756worker1 # [ 127.544679] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5757worker1 # [ 127.549408] nix-daemon[834]: reaped child process 1437, status = succeeded5758client # [ 127.556962] nix[1339]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5759worker1 # [ 127.561656] nix-daemon[834]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5760lb # [ 128.789335] envoy[898]: conn peer=[2001:db8:1::1]:53310 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9550 tx=4083 ms=55275761lb: must succeed: curl -sf localhost:9901/clusters5762lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5763lb: must succeed: curl -sf localhost:9901/clusters5764lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5765lb: must succeed: curl -sf localhost:9901/clusters5766lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5767lb: must succeed: curl -sf localhost:9901/clusters5768lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5769worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5770worker2 # [ 129.683815] systemd-journald[269]: Received client request to rotate journal, rotating.5771worker2 # [ 129.704452] systemd-journald[269]: Vacuuming done, freed 0B of archived journals from /var/log/journal/4bc15a8868284ebaa5af924a396283cb.5772worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds)5773worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5774worker1 # [ 129.702180] systemd[1]: Stopping Nix worker-protocol over gRPC...5775worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5776client: must succeed: sleep 5.05777worker1 # [ 130.227512] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:35Z level=info event=draining5778client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5779worker1 # [ 134.990128] nix-daemon[834]: reaped child process 1451, status = succeeded5780worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5781worker1 # [ 135.088592] nix-daemon[834]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)5782worker1 # [ 135.111703] nix-daemon[834]: reaped child process 1495, status = succeeded5783lb # [ 135.134318] niks3-server[1138]: 2026/09/22 11:00:40 INFO Received uploads request method=POST path=/api/pending_closures5784worker1 # [ 135.124187] nix-grpc-daemon[1308]: time=2026-09-22T11:00:40.716Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5785worker1 # [ 135.125686] nix-grpc-daemon[1308]: time=2026-09-22T11:00:40.716Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5786lb # [ 135.177310] niks3-server[1138]: 2026/09/22 11:00:40 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5787lb # [ 135.189706] niks3-server[1138]: 2026/09/22 11:00:40 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5788lb # [ 135.203382] niks3-server[1138]: 2026/09/22 11:00:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5789worker1 # [ 135.182837] nix-grpc-daemon[1308]: time=2026-09-22T11:00:40.777Z level=INFO msg="Uploading 1 narinfos"5790lb # [ 135.211038] niks3-server[1138]: 2026/09/22 11:00:40 INFO Signed narinfos id=44 count=15791lb # [ 135.218044] niks3-server[1138]: 2026/09/22 11:00:40 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5792lb # [ 135.227315] niks3-server[1138]: 2026/09/22 11:00:40 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5793lb # [ 135.233800] niks3-server[1138]: 2026/09/22 11:00:40 INFO Completed upload id=445794worker1 # [ 135.212107] nix-grpc-daemon[1308]: time=2026-09-22T11:00:40.807Z level=INFO msg="Upload complete. (235ms)"5795lb # [ 135.238364] niks3-server[1138]: 2026/09/22 11:00:40 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5796worker1 # [ 135.227282] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47014 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15797worker1 # [ 135.244589] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:40Z level=info event=signal_exit5798worker1 # [ 135.248792] nix-daemon[834]: reaped child process 1449, status = succeeded5799client # [ 135.259820] systemd[1]: rs.service: Deactivated successfully.5800client # [ 135.264276] systemd[1]: rs.service: Consumed 195ms CPU time over 8.444s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.5801lb # [ 135.552238] niks3-server[1138]: 2026/09/22 11:00:41 INFO lead: released remote=192.168.1.3:360745802lb # [ 136.427492] niks3-server[1138]: 2026/09/22 11:00:42 INFO lead: acquired remote=192.168.1.4:421285803worker2 # [ 136.449486] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_take_over5804worker2 # [ 136.574404] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_restarting addr=lb:500515805worker1 # [ 136.536836] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:42Z level=info event=scheduler_restarting5806worker1 # [ 136.542167] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:42Z level=info event=scheduler_restarting addr=lb:500515807worker1 # [ 136.547109] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386665808worker1 # [ 136.552207] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:42Z level=info event=worker_gone addr=192.168.1.3:500515809worker1 # [ 136.557125] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386665810worker1 # [ 136.562418] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:42Z level=info event=worker_gone addr=192.168.1.4:500515811worker1 # [ 136.587787] nix-daemon[834]: reaped child process 1278, status = succeeded5812worker2 # [ 136.776242] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_connected addr=lb:500515813worker2 # [ 136.781684] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5814worker1 # [ 136.755085] nix-grpc-daemon[1226]: ts=2026-09-22T11:00:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5815worker2 # [ 136.982766] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_connected addr=lb:500515816worker2 # [ 136.988111] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5817worker1 # [ 137.089051] nix-grpc-daemon[1226]: E0000 00:00:1790074842.683901 1271 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5818worker1 # [ 137.096064] nix-grpc-daemon[1226]: E0000 00:00:1790074842.684172 1271 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5819worker2 # [ 137.189643] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_connected addr=lb:500515820worker2 # [ 137.199722] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5821worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.19 seconds)5822client: waiting for success: ! systemctl is-active rs5823client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5824client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5825client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5826client: must fail: journalctl -u rs | grep -q 'reconnecting'5827client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5828lb: must succeed: curl -sf localhost:9901/clusters5829worker2 # [ 137.398148] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:42Z level=info event=scheduler_connected addr=lb:500515830worker2 # [ 137.403500] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5831lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5832worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5833worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5834worker2 # [ 137.606609] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=scheduler_connected addr=lb:500515835worker2 # [ 137.610824] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5836worker2 # [ 137.807518] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=scheduler_connected addr=lb:500515837worker2 # [ 137.812819] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5838worker2 # [ 138.013855] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=scheduler_connected addr=lb:500515839worker2 # [ 138.031795] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=oidc_jwks provider=mock keys=15840worker2 # [ 138.036382] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:534445841worker2 # [ 138.041645] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05842lb: must succeed: curl -sf localhost:9901/clusters5843lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5844worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5845worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5846worker1 # [ 138.623494] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5847worker1 # [ 138.630379] systemd[1]: Stopped Nix worker-protocol over gRPC.5848worker1 # [ 138.635949] systemd[1]: nix-grpc-daemon.service: Consumed 1.950s CPU time over 32.566s wall clock time, 38.7M memory peak, 47.4K incoming IP traffic, 93.2K outgoing IP traffic.5849worker1 # [ 138.652190] systemd[1]: Starting Nix worker-protocol over gRPC...5850lb # [ 138.792628] envoy[898]: conn peer=[2001:db8:1::1]:52380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1427 ms=82145851worker1 # [ 138.794415] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5852worker1 # [ 138.796741] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5853worker1 # [ 138.800528] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15854worker1 # [ 138.807054] nix-grpc-daemon[1502]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5855worker1 # [ 138.820288] nix-grpc-daemon[1502]: E0000 00:00:1790074844.400123 1502 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5856worker1 # [ 138.847664] nix-grpc-daemon[1502]: E0000 00:00:1790074844.402138 1502 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5857worker1 # [ 138.851036] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5858worker1 # [ 138.853311] nix-grpc-daemon[1502]: E0000 00:00:1790074844.410069 1541 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5859worker1 # [ 138.856249] nix-grpc-daemon[1502]: E0000 00:00:1790074844.428183 1541 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5860worker1 # [ 138.862097] nix-grpc-daemon[1502]: E0000 00:00:1790074844.428349 1541 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5861worker1 # [ 138.866921] nix-grpc-daemon[1502]: E0000 00:00:1790074844.442791 1541 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5862worker1 # [ 138.869872] systemd[1]: Started Nix worker-protocol over gRPC.5863worker1 # [ 138.870739] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:44Z level=info event=scheduler_connected addr=lb:500515864worker2 # [ 138.912638] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:534445865worker2 # [ 138.917560] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05866lb # [ 138.797635] envoy[898]: conn peer=[2001:db8:1::3]:41884 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2413 tx=836 ms=304735867lb: must succeed: curl -sf localhost:9901/clusters5868lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5869worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5870worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5871(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.99 seconds)5872subtest: scheduler restart mid-build: build finishes, no double build5873client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5874client # [ 139.808784] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5875client # Running as unit: mid.service; invocation ID: 5a0f2b633e43426d9afabde0c9048a0c5876client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5877worker1 # [ 140.010918] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:348825878worker1 # [ 140.025626] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)5879client # [ 140.041929] nix[1375]: don't know how to build these paths:5880client # [ 140.043890] nix[1375]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5881worker2 # [ 140.103963] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:45Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05882worker2 # [ 140.109343] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:45Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15883worker2 # [ 140.163322] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534545884worker2 # [ 140.177948] nix-daemon[855]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5885worker2 # [ 140.200790] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)5886worker2 # [ 140.269541] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 187 ms (attempt 2/5)5887worker2 # [ 140.458596] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 3/5)5888worker2 # [ 140.476239] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 727 ms (attempt 4/5)5889worker2 # [ 141.213603] nix-grpc-daemon[1442]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5890worker2 # [ 141.219195] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5891worker2 # [ 141.239357] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534545892client # [ 141.208627] nix[1375]: copying 1 paths to 'lb:50051'...5893worker2 # [ 141.256432] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:534545894client # [ 141.214420] nix[1375]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5895worker2 # [ 141.271727] nix-daemon[855]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5896worker2 # [ 141.298372] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:46Z level=info event=niks3_push_started pid=15715897worker2 # [ 141.445194] nix-grpc-daemon[1571]: time=2026-09-22T11:00:47.043Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5898worker2 # [ 141.551258] nix-daemon[855]: accepted connection from pid 1584, user nix-grpc-daemon (trusted)5899worker2 # [ 141.570465] nix-daemon[855]: reaped child process 1587, status = succeeded5900worker2 # [ 141.672214] nix-daemon[855]: accepted connection from pid 1589, user nix-grpc-daemon (trusted)5901worker2 # [ 141.689518] nix-daemon[855]: reaped child process 1592, status = succeeded5902lb # [ 141.673005] niks3-server[1138]: 2026/09/22 11:00:47 INFO Received uploads request method=POST path=/api/pending_closures5903worker2 # [ 141.704094] nix-grpc-daemon[1571]: time=2026-09-22T11:00:47.302Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5904worker2 # [ 141.705551] nix-grpc-daemon[1571]: time=2026-09-22T11:00:47.302Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5905lb # [ 141.710387] niks3-server[1138]: 2026/09/22 11:00:47 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5906worker2 # [ 141.753076] nix-grpc-daemon[1571]: time=2026-09-22T11:00:47.351Z level=INFO msg="Uploading 1 narinfos"5907lb # [ 141.730845] niks3-server[1138]: 2026/09/22 11:00:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5908lb # [ 141.743787] niks3-server[1138]: 2026/09/22 11:00:47 INFO Signed narinfos id=45 count=15909lb # [ 141.756260] niks3-server[1138]: 2026/09/22 11:00:47 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5910lb # [ 141.765872] niks3-server[1138]: 2026/09/22 11:00:47 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5911lb # [ 141.777095] niks3-server[1138]: 2026/09/22 11:00:47 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5912lb # [ 141.785519] niks3-server[1138]: 2026/09/22 11:00:47 INFO Completed upload id=455913worker2 # [ 141.805651] nix-grpc-daemon[1571]: time=2026-09-22T11:00:47.404Z level=INFO msg="Upload complete. (356ms)"5914worker2 # [ 141.820339] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53454 duration_s=0 paths=1 nar_bytes_in=4245915worker2 # [ 141.832518] nix-grpc-daemon[1442]: ts=2026-09-22T11:00:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534545916worker2 # [ 141.841860] nix-daemon[855]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5917worker2 # [ 141.847365] nix-daemon[855]: reaped child process 1569, status = succeeded5918client # [ 141.812581] nix[1375]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5919worker2 # [ 141.860415] nix-daemon[855]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5920lb: must succeed: curl -sf localhost:9901/clusters5921lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5922lb: must succeed: curl -sf localhost:9901/clusters5923lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5924worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5925worker2 # [ 143.720809] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1442 (nix-grpc-daemon) on client request.5926worker2 # [ 143.722404] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5927lb # [ 143.720842] niks3-server[1138]: 2026/09/22 11:00:49 INFO lead: released remote=192.168.1.4:421285928worker2 # [ 143.749613] systemd[1]: Starting scheduler-token.service...5929worker2 # [ 143.750453] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5930worker2 # [ 143.751869] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5931client # [ 143.705253] nix[1375]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5932worker1 # [ 143.707022] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:49Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5933client # [ 143.711922] nix[1375]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5934worker2 # [ 143.769851] systemd[1]: nix-grpc-daemon.service: Consumed 594ms CPU time over 17.448s wall clock time, 34.3M memory peak, 16.6K incoming IP traffic, 31.6K outgoing IP traffic.5935worker2 # [ 143.805671] nix-daemon[855]: reaped child process 1559, status = succeeded5936worker2 # [ 143.808759] nix-daemon[855]: reaped child process 1594, status = succeeded5937worker2 # [ 143.817721] nix-daemon[1596]: unexpected Nix daemon error: error: interrupted by the user5938worker2 # [ 143.825869] nix-daemon[855]: reaped child process 1596, status = failed with exit code 15939worker2 # [ 143.842304] systemd[1]: scheduler-token.service: Deactivated successfully.5940worker2 # [ 143.843297] systemd[1]: Finished scheduler-token.service.5941worker2 # [ 143.845788] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 90ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5942worker2 # [ 143.851950] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5943worker2 # [ 143.853664] systemd[1]: Starting scheduler-token.service...5944lb # [ 143.838468] niks3-server[1138]: 2026/09/22 11:00:49 INFO lead: acquired remote=192.168.1.3:527985945worker1 # [ 143.821687] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:49Z level=info event=scheduler_take_over5946worker2 # [ 143.896807] systemd[1]: scheduler-token.service: Deactivated successfully.5947worker2 # [ 143.899210] systemd[1]: Finished scheduler-token.service.5948worker2 # [ 143.900003] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 52ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5949worker2 # [ 143.905217] systemd[1]: Starting Nix worker-protocol over gRPC...5950worker1 # [ 143.907200] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:49Z level=info event=scheduler_connected addr=lb:500515951worker2 # [ 144.005789] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5952worker2 # [ 144.007956] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5953worker2 # [ 144.011833] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15954worker2 # [ 144.017330] nix-grpc-daemon[1639]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5955worker2 # [ 144.018761] nix-grpc-daemon[1639]: E0000 00:00:1790074849.615834 1639 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5956worker1 # [ 143.998357] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5957worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.37 seconds)5958client: must succeed: sleep 5.05959worker2 # [ 144.064191] nix-grpc-daemon[1639]: E0000 00:00:1790074849.621827 1639 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5960worker2 # [ 144.073969] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5961worker2 # [ 144.076136] nix-grpc-daemon[1639]: E0000 00:00:1790074849.631717 1678 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5962worker2 # [ 144.078626] nix-grpc-daemon[1639]: E0000 00:00:1790074849.654318 1678 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5963worker2 # [ 144.083579] nix-grpc-daemon[1639]: E0000 00:00:1790074849.654658 1678 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5964worker2 # [ 144.088485] systemd[1]: Started Nix worker-protocol over gRPC.5965worker2 # [ 144.089330] nix-grpc-daemon[1639]: E0000 00:00:1790074849.672595 1678 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5966worker2 # [ 144.091645] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:49Z level=info event=scheduler_connected addr=lb:500515967worker2 # [ 144.093000] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5968worker1 # [ 144.199258] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:49Z level=info event=scheduler_connected addr=lb:500515969worker1 # [ 144.204675] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5970client # [ 144.208463] nix[1375]: scheduler at lb:50051: no healthy upstream, reconnecting5971worker2 # [ 144.290076] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:49Z level=info event=scheduler_connected addr=lb:500515972worker2 # [ 144.295018] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5973worker1 # [ 144.405581] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515974worker1 # [ 144.410845] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5975worker2 # [ 144.496337] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515976worker2 # [ 144.501587] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5977worker1 # [ 144.611666] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515978worker1 # [ 144.617021] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5979worker2 # [ 144.702732] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515980worker2 # [ 144.708282] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5981worker1 # [ 144.819514] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515982worker1 # [ 144.823958] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5983worker2 # [ 144.909476] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515984worker2 # [ 144.914733] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5985worker1 # [ 145.020126] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515986worker1 # [ 145.025236] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5987worker2 # [ 145.118135] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515988worker2 # [ 145.122447] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5989client # [ 145.212219] nix[1375]: scheduler at lb:50051: no healthy upstream, reconnecting5990worker1 # [ 145.226292] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515991worker1 # [ 145.231455] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5992worker2 # [ 145.318633] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_connected addr=lb:500515993worker2 # [ 145.324249] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5994worker1 # [ 145.432472] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=scheduler_connected addr=lb:500515995worker1 # [ 145.437741] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5996worker2 # [ 145.525554] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:51Z level=info event=scheduler_connected addr=lb:500515997worker2 # [ 145.530993] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5998worker1 # [ 145.638504] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=scheduler_connected addr=lb:500515999worker1 # [ 145.643839] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6000worker2 # [ 145.732171] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:51Z level=info event=scheduler_connected addr=lb:500516001worker1 # [ 145.711676] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=oidc_jwks provider=mock keys=16002worker1 # [ 145.716381] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:520426003worker1 # [ 145.721663] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06004worker1 # [ 145.847321] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=scheduler_connected addr=lb:500516005worker1 # [ 145.851534] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:520426006worker1 # [ 145.856512] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06007worker1 # [ 147.225465] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:52Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06008worker1 # [ 147.225854] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:52Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16009worker2 # [ 147.320881] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434046010worker2 # [ 147.336216] nix-daemon[855]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6011worker2 # [ 147.359994] nix-daemon[855]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6012worker2 # [ 147.376351] nix-daemon[855]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6013client # [ 147.330911] nix[1375]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6014lb # [ 148.792966] envoy[898]: conn peer=[2001:db8:1::4]:40216 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6484 tx=562 ms=172576015client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6016client: waiting for success: ! systemctl is-active mid6017worker2 # [ 149.401922] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:54Z level=info event=niks3_push_started pid=17206018worker2 # [ 149.449400] nix-daemon[855]: reaped child process 1695, status = succeeded6019worker2 # [ 149.519533] nix-grpc-daemon[1720]: time=2026-09-22T11:00:55.117Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6020worker2 # [ 149.630493] nix-daemon[855]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6021worker2 # [ 149.649665] nix-daemon[855]: reaped child process 1736, status = succeeded6022lb # [ 149.633741] niks3-server[1138]: 2026/09/22 11:00:55 INFO Received uploads request method=POST path=/api/pending_closures6023worker2 # [ 149.666380] nix-grpc-daemon[1720]: time=2026-09-22T11:00:55.263Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6024worker2 # [ 149.667866] nix-grpc-daemon[1720]: time=2026-09-22T11:00:55.263Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6025lb # [ 149.676316] niks3-server[1138]: 2026/09/22 11:00:55 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6026lb # [ 149.693037] niks3-server[1138]: 2026/09/22 11:00:55 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6027worker2 # [ 149.730241] nix-grpc-daemon[1720]: time=2026-09-22T11:00:55.328Z level=INFO msg="Uploading 1 narinfos"6028lb # [ 149.707240] niks3-server[1138]: 2026/09/22 11:00:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6029lb # [ 149.714219] niks3-server[1138]: 2026/09/22 11:00:55 INFO Signed narinfos id=46 count=16030lb # [ 149.724482] niks3-server[1138]: 2026/09/22 11:00:55 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6031lb # [ 149.735169] niks3-server[1138]: 2026/09/22 11:00:55 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6032worker2 # [ 149.763096] nix-grpc-daemon[1720]: time=2026-09-22T11:00:55.362Z level=INFO msg="Upload complete. (240ms)"6033lb # [ 149.744176] niks3-server[1138]: 2026/09/22 11:00:55 INFO Completed upload id=466034worker2 # [ 149.776347] nix-grpc-daemon[1639]: ts=2026-09-22T11:00:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43404 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16035worker2 # [ 149.792370] nix-daemon[855]: reaped child process 1693, status = succeeded6036client # [ 149.764776] systemd[1]: mid.service: Deactivated successfully.6037client # [ 149.769749] systemd[1]: mid.service: Consumed 205ms CPU time over 9.957s wall clock time, 39.2M memory peak, 9.1K incoming IP traffic, 11.3K outgoing IP traffic.6038client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6039client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6040client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)6041(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.76 seconds)6042subtest: interrupting the client stops the build on the worker6043client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6044client # [ 150.539067] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6045client # Running as unit: intr.service; invocation ID: e6d7d442468541668474c2b2717972e96046client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6047client # [ 150.719716] nix[1411]: don't know how to build these paths:6048client # [ 150.720896] nix[1411]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6049worker1 # [ 150.721493] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:348826050worker1 # [ 150.735293] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:56Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06051worker1 # [ 150.742169] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:56Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=16052worker1 # [ 150.783467] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826053worker1 # [ 150.798941] nix-grpc-daemon[1502]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5)6054worker1 # [ 150.893479] nix-grpc-daemon[1502]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 2/5)6055worker1 # [ 151.026628] nix-grpc-daemon[1502]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 257 ms (attempt 3/5)6056worker1 # [ 151.285749] nix-grpc-daemon[1502]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 652 ms (attempt 4/5)6057worker1 # [ 151.941223] nix-grpc-daemon[1502]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6058worker1 # [ 151.946733] nix-grpc-daemon[1502]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6059worker1 # [ 151.960995] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826060client # [ 151.978512] nix[1411]: copying 1 paths to 'lb:50051'...6061worker1 # [ 151.983705] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826062client # [ 151.984149] nix[1411]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6063worker1 # [ 152.001959] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6064worker1 # [ 152.029923] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:57Z level=info event=niks3_push_started pid=16026065worker1 # [ 152.178342] nix-grpc-daemon[1602]: time=2026-09-22T11:00:57.772Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6066worker1 # [ 152.284906] nix-daemon[834]: accepted connection from pid 1615, user nix-grpc-daemon (trusted)6067worker1 # [ 152.307241] nix-daemon[834]: reaped child process 1618, status = succeeded6068worker1 # [ 152.408110] nix-daemon[834]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6069worker1 # [ 152.425439] nix-daemon[834]: reaped child process 1623, status = succeeded6070lb # [ 152.452344] niks3-server[1138]: 2026/09/22 11:00:58 INFO Received uploads request method=POST path=/api/pending_closures6071worker1 # [ 152.446061] nix-grpc-daemon[1602]: time=2026-09-22T11:00:58.040Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6072worker1 # [ 152.447557] nix-grpc-daemon[1602]: time=2026-09-22T11:00:58.040Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6073lb # [ 152.501867] niks3-server[1138]: 2026/09/22 11:00:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6074worker1 # [ 152.488505] nix-grpc-daemon[1602]: time=2026-09-22T11:00:58.083Z level=INFO msg="Uploading 1 narinfos"6075lb # [ 152.517235] niks3-server[1138]: 2026/09/22 11:00:58 INFO Signed narinfos id=47 count=16076lb # [ 152.519871] niks3-server[1138]: 2026/09/22 11:00:58 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6077lb # [ 152.531925] niks3-server[1138]: 2026/09/22 11:00:58 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6078lb # [ 152.540598] niks3-server[1138]: 2026/09/22 11:00:58 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6079lb # [ 152.547232] niks3-server[1138]: 2026/09/22 11:00:58 INFO Completed upload id=476080worker1 # [ 152.526506] nix-grpc-daemon[1602]: time=2026-09-22T11:00:58.121Z level=INFO msg="Upload complete. (345ms)"6081lb # [ 152.549932] niks3-server[1138]: 2026/09/22 11:00:58 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6082worker1 # [ 152.540435] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=4246083worker1 # [ 152.551216] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826084worker1 # [ 152.560634] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6085worker1 # [ 152.568396] nix-daemon[834]: reaped child process 1600, status = succeeded6086client # [ 152.570234] nix[1411]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6087worker1 # [ 152.574881] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6088client: must succeed: systemctl kill -s INT intr6089worker1 # [ 154.183116] nix-grpc-daemon[1502]: ts=2026-09-22T11:00:59Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6090client # [ 154.181220] nix[1411]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6091client # [ 154.182153] systemd[1]: intr.service: Sent signal SIGINT to main process 1411 (nix) on client request.6092worker1 # [ 154.201129] nix-daemon[834]: reaped child process 1625, status = succeeded6093client # [ 154.200243] nix[1411]: error: interrupted by the user6094worker1 # [ 154.203515] nix-daemon[1627]: unexpected Nix daemon error: error: interrupted by the user6095client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6096worker1 # [ 154.212692] nix-daemon[834]: reaped child process 1627, status = failed with exit code 16097client # [ 154.231845] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6098client # [ 154.237151] systemd[1]: intr.service: Failed with result 'exit-code'.6099client # [ 154.239645] systemd[1]: intr.service: Consumed 174ms CPU time over 3.696s wall clock time, 41.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.6100(finished: subtest: interrupting the client stops the build on the worker, in 3.81 seconds)6101subtest: a deploy mid-build drains: the build finishes, the new generation takes over6102client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6103client # [ 154.372099] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6104client # Running as unit: sw.service; invocation ID: f694eca68945446d98ab0c24acb806c16105client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)6106client # [ 154.564212] nix[1432]: don't know how to build these paths:6107client # [ 154.565174] nix[1432]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6108worker1 # [ 154.564325] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:348826109worker1 # [ 154.577736] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:00Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06110worker1 # [ 154.583864] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:00Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=16111worker2 # [ 154.667678] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434046112worker2 # [ 154.685046] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)6113worker2 # [ 154.701067] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 2/5)6114worker2 # [ 154.756963] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 3/5)6115worker2 # [ 154.883950] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 791 ms (attempt 4/5)6116worker2 # [ 155.681464] nix-grpc-daemon[1639]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6117worker2 # [ 155.685683] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6118worker2 # [ 155.706107] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434046119client # [ 155.677148] nix[1432]: copying 1 paths to 'lb:50051'...6120worker2 # [ 155.722055] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:434046121client # [ 155.681634] nix[1432]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6122worker2 # [ 155.742998] nix-daemon[855]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6123worker2 # [ 155.891883] nix-daemon[855]: accepted connection from pid 1772, user nix-grpc-daemon (trusted)6124worker2 # [ 155.911727] nix-daemon[855]: reaped child process 1775, status = succeeded6125worker2 # [ 156.013888] nix-daemon[855]: accepted connection from pid 1777, user nix-grpc-daemon (trusted)6126worker2 # [ 156.031170] nix-daemon[855]: reaped child process 1780, status = succeeded6127lb # [ 156.017743] niks3-server[1138]: 2026/09/22 11:01:01 INFO Received uploads request method=POST path=/api/pending_closures6128worker2 # [ 156.052152] nix-grpc-daemon[1720]: time=2026-09-22T11:01:01.650Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6129worker2 # [ 156.053610] nix-grpc-daemon[1720]: time=2026-09-22T11:01:01.650Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6130lb # [ 156.056497] niks3-server[1138]: 2026/09/22 11:01:01 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6131lb # [ 156.069658] niks3-server[1138]: 2026/09/22 11:01:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6132worker2 # [ 156.096111] nix-grpc-daemon[1720]: time=2026-09-22T11:01:01.695Z level=INFO msg="Uploading 1 narinfos"6133lb # [ 156.080220] niks3-server[1138]: 2026/09/22 11:01:01 INFO Signed narinfos id=48 count=16134lb # [ 156.082263] niks3-server[1138]: 2026/09/22 11:01:01 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6135lb # [ 156.094110] niks3-server[1138]: 2026/09/22 11:01:01 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6136lb # [ 156.100179] niks3-server[1138]: 2026/09/22 11:01:01 INFO Completed upload id=486137lb # [ 156.102740] niks3-server[1138]: 2026/09/22 11:01:01 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6138worker2 # [ 156.121115] nix-grpc-daemon[1720]: time=2026-09-22T11:01:01.720Z level=INFO msg="Upload complete. (352ms)"6139worker2 # [ 156.131833] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43404 duration_s=0 paths=1 nar_bytes_in=4166140worker2 # [ 156.143429] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434046141worker2 # [ 156.150312] nix-daemon[855]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6142worker2 # [ 156.154967] nix-daemon[855]: reaped child process 1770, status = succeeded6143worker2 # [ 156.165965] nix-daemon[855]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6144client # [ 156.120160] nix[1432]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6145worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26146worker2 # Checking switch inhibitors... done6147worker2 # [ 157.518071] nixos[1812]: switching to system configuration /nix/store/qrk4msql1qyv8s1y26xr9w72x0zgq22p-nixos-system-worker2-test6148worker2 # activating the configuration...6149worker2 # [ 157.525090] systemd[1]: Stopped target Remote File Systems.6150worker2 # [ 157.525966] systemd[1]: Stopped target Local File Systems.6151worker2 # setting up /etc...6152worker2 # [ 158.089430] systemd[1]: Reload requested from client PID 1812 ('.switch-to-conf') (unit backdoor.service)...6153worker2 # [ 158.091515] systemd[1]: Reloading...6154lb # [ 158.790849] envoy[898]: conn peer=[2001:db8:1::1]:34752 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4301 tx=1976 ms=97436155lb # [ 158.795948] envoy[898]: conn peer=[2001:db8:1::1]:52174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=849 ms=34946156worker2 # [ 163.004821] systemd[1]: Reloading finished in 4912 ms.6157worker2 # restarting sysinit-reactivation.target6158worker2 # [ 163.123265] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6159worker2 # [ 163.243725] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6160worker2 # [ 163.245346] systemd[1]: Reached target Reactivate sysinit units.6161worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6162worker2 # [ 163.249140] systemd[1]: Reloading D-Bus System Message Bus...6163worker2 # [ 163.250311] dbus-broker-launch[491]: Caught SIGHUP, trigger reload.6164worker2 # [ 163.262312] systemd[1]: Reloading Nix worker-protocol over gRPC...6165worker2 # [ 163.271436] dbus-broker-launch[491]: Looking up NSS user entry for 'systemd-timesync'...6166worker2 # [ 163.277303] dbus-broker-launch[491]: NSS returned no entry for 'systemd-timesync'6167worker2 # [ 163.291393] dbus-broker-launch[491]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6168worker2 # [ 163.303960] systemd[1]: Reloaded Nix worker-protocol over gRPC.6169worker2 # [ 163.311317] systemd[1]: Reloaded D-Bus System Message Bus.6170worker2 # [ 163.319148] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6171worker2 # [ 163.324207] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6172worker2 # [ 163.325487] systemd[1]: Reached target Remote File Systems.6173worker2 # [ 163.336361] systemd[1]: Starting linger-users.service...6174worker2 # [ 163.341716] systemd[1]: Reached target Local File Systems.6175worker2 # [ 163.342581] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6176worker2 # [ 163.356552] systemd[1]: Starting Create SUID/SGID Wrappers...6177worker2 # [ 163.357417] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6178worker2 # [ 163.358613] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676179worker2 # [ 163.370435] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6180worker2 # [ 163.380199] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6181worker2 # [ 163.384758] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6182worker2 # [ 163.407347] systemd[1]: linger-users.service: Deactivated successfully.6183worker2 # [ 163.408442] systemd[1]: Finished linger-users.service.6184worker2 # [ 163.698823] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6185worker2 # [ 163.699947] systemd[1]: Finished Create SUID/SGID Wrappers.6186worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6187worker2 # [ 163.956699] nixos[1812]: finished switching to system configuration /nix/store/qrk4msql1qyv8s1y26xr9w72x0zgq22p-nixos-system-worker2-test6188worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.89 seconds)6189worker2: must succeed: pgrep -f 'read -t [9]0 x'6190worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6191client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26192worker2 # [ 164.037160] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:09Z level=info event=draining6193client # don't know how to build these paths:6194client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6195worker1 # [ 164.205498] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:348826196worker1 # [ 164.209918] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826197worker1 # [ 164.229629] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:09Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06198worker1 # [ 164.232802] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:09Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16199worker1 # [ 164.280670] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826200worker1 # [ 164.297255] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826201client # copying 1 paths to 'lb:50051'...6202worker1 # [ 164.315059] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826203client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6204worker1 # [ 164.338282] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6205worker1 # [ 164.511934] nix-daemon[834]: accepted connection from pid 1678, user nix-grpc-daemon (trusted)6206worker1 # [ 164.536640] nix-daemon[834]: reaped child process 1681, status = succeeded6207worker1 # [ 164.642462] nix-daemon[834]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6208worker1 # [ 164.663743] nix-daemon[834]: reaped child process 1686, status = succeeded6209lb # [ 164.689051] niks3-server[1138]: 2026/09/22 11:01:10 INFO Received uploads request method=POST path=/api/pending_closures6210worker1 # [ 164.678274] nix-grpc-daemon[1602]: time=2026-09-22T11:01:10.272Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6211worker1 # [ 164.679773] nix-grpc-daemon[1602]: time=2026-09-22T11:01:10.272Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6212lb # [ 164.719465] niks3-server[1138]: 2026/09/22 11:01:10 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6213lb # [ 164.734542] niks3-server[1138]: 2026/09/22 11:01:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6214worker1 # [ 164.720298] nix-grpc-daemon[1602]: time=2026-09-22T11:01:10.315Z level=INFO msg="Uploading 1 narinfos"6215lb # [ 164.746491] niks3-server[1138]: 2026/09/22 11:01:10 INFO Signed narinfos id=49 count=16216lb # [ 164.752393] niks3-server[1138]: 2026/09/22 11:01:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6217lb # [ 164.762624] niks3-server[1138]: 2026/09/22 11:01:10 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6218worker1 # [ 164.749956] nix-grpc-daemon[1602]: time=2026-09-22T11:01:10.343Z level=INFO msg="Upload complete. (392ms)"6219lb # [ 164.773190] niks3-server[1138]: 2026/09/22 11:01:10 INFO Completed upload id=496220lb # [ 164.778044] niks3-server[1138]: 2026/09/22 11:01:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6221worker1 # [ 164.755697] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=5286222worker1 # [ 164.770715] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826223worker1 # [ 164.777683] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6224worker1 # [ 164.782677] nix-daemon[834]: reaped child process 1676, status = succeeded6225client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6226worker1 # [ 164.793468] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6227client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6228client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6229client # farm-a> LOG-farm-a-during-drain6230worker1 # [ 166.068729] nix-daemon[834]: reaped child process 1690, status = succeeded6231worker1 # [ 166.175519] nix-daemon[834]: accepted connection from pid 1706, user nix-grpc-daemon (trusted)6232worker1 # [ 166.194511] nix-daemon[834]: reaped child process 1709, status = succeeded6233lb # [ 166.224564] niks3-server[1138]: 2026/09/22 11:01:11 INFO Received uploads request method=POST path=/api/pending_closures6234worker1 # [ 166.215615] nix-grpc-daemon[1602]: time=2026-09-22T11:01:11.807Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6235worker1 # [ 166.218825] nix-grpc-daemon[1602]: time=2026-09-22T11:01:11.807Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6236lb # [ 166.267754] niks3-server[1138]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6237lb # [ 166.278884] niks3-server[1138]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6238worker1 # [ 166.271921] nix-grpc-daemon[1602]: time=2026-09-22T11:01:11.865Z level=INFO msg="Uploading 1 narinfos"6239lb # [ 166.292623] niks3-server[1138]: 2026/09/22 11:01:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6240lb # [ 166.302150] niks3-server[1138]: 2026/09/22 11:01:11 INFO Signed narinfos id=50 count=16241lb # [ 166.308226] niks3-server[1138]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6242lb # [ 166.318607] niks3-server[1138]: 2026/09/22 11:01:11 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6243lb # [ 166.323729] niks3-server[1138]: 2026/09/22 11:01:11 INFO Completed upload id=506244worker1 # [ 166.302185] nix-grpc-daemon[1602]: time=2026-09-22T11:01:11.897Z level=INFO msg="Upload complete. (296ms)"6245lb # [ 166.328559] niks3-server[1138]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6246worker1 # [ 166.321603] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16247worker1 # [ 166.338804] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:11Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06248worker1 # [ 166.343319] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:11Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16249worker1 # [ 166.347823] nix-daemon[834]: reaped child process 1688, status = succeeded6250worker1 # [ 166.374537] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826251worker1 # [ 166.390124] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826252worker1 # [ 166.406063] nix-grpc-daemon[1602]: time=2026-09-22T11:01:12.001Z level=INFO msg="All 1 paths already cached"6253client # copying 1 paths to 'lb:50051'...6254worker1 # [ 166.416779] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826255client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6256worker1 # [ 166.439868] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6257worker1 # [ 166.598718] nix-daemon[834]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6258worker1 # [ 166.620948] nix-daemon[834]: reaped child process 1718, status = succeeded6259worker1 # [ 166.723276] nix-daemon[834]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6260worker1 # [ 166.740385] nix-daemon[834]: reaped child process 1723, status = succeeded6261lb # [ 166.766527] niks3-server[1138]: 2026/09/22 11:01:12 INFO Received uploads request method=POST path=/api/pending_closures6262worker1 # [ 166.754854] nix-grpc-daemon[1602]: time=2026-09-22T11:01:12.349Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6263worker1 # [ 166.756438] nix-grpc-daemon[1602]: time=2026-09-22T11:01:12.349Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6264lb # [ 166.796063] niks3-server[1138]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6265lb # [ 166.811186] niks3-server[1138]: 2026/09/22 11:01:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6266worker1 # [ 166.798473] nix-grpc-daemon[1602]: time=2026-09-22T11:01:12.393Z level=INFO msg="Uploading 1 narinfos"6267lb # [ 166.824997] niks3-server[1138]: 2026/09/22 11:01:12 INFO Signed narinfos id=51 count=16268lb # [ 166.827792] niks3-server[1138]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6269lb # [ 166.842210] niks3-server[1138]: 2026/09/22 11:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6270lb # [ 166.848931] niks3-server[1138]: 2026/09/22 11:01:12 INFO Completed upload id=516271worker1 # [ 166.827546] nix-grpc-daemon[1602]: time=2026-09-22T11:01:12.422Z level=INFO msg="Upload complete. (370ms)"6272lb # [ 166.852570] niks3-server[1138]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6273worker1 # [ 166.836342] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=6406274worker1 # [ 166.851736] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826275worker1 # [ 166.861760] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6276worker1 # [ 166.864861] nix-daemon[834]: reaped child process 1713, status = succeeded6277client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6278client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6279worker1 # [ 166.879022] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6280client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6281client # farm-b> LOG-farm-b-during-drain6282worker1 # [ 167.790308] nix-daemon[834]: reaped child process 1727, status = succeeded6283worker1 # [ 167.865745] nix-daemon[834]: accepted connection from pid 1743, user nix-grpc-daemon (trusted)6284worker1 # [ 167.887594] nix-daemon[834]: reaped child process 1746, status = succeeded6285lb # [ 167.910145] niks3-server[1138]: 2026/09/22 11:01:13 INFO Received uploads request method=POST path=/api/pending_closures6286worker1 # [ 167.899509] nix-grpc-daemon[1602]: time=2026-09-22T11:01:13.493Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6287worker1 # [ 167.901659] nix-grpc-daemon[1602]: time=2026-09-22T11:01:13.493Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6288lb # [ 167.955476] niks3-server[1138]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6289lb # [ 167.967345] niks3-server[1138]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6290worker1 # [ 167.952875] nix-grpc-daemon[1602]: time=2026-09-22T11:01:13.547Z level=INFO msg="Uploading 1 narinfos"6291lb # [ 167.975644] niks3-server[1138]: 2026/09/22 11:01:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6292lb # [ 167.984983] niks3-server[1138]: 2026/09/22 11:01:13 INFO Signed narinfos id=52 count=16293lb # [ 167.990623] niks3-server[1138]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6294lb # [ 167.997728] niks3-server[1138]: 2026/09/22 11:01:13 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6295worker1 # [ 167.982860] nix-grpc-daemon[1602]: time=2026-09-22T11:01:13.578Z level=INFO msg="Upload complete. (293ms)"6296lb # [ 168.006948] niks3-server[1138]: 2026/09/22 11:01:13 INFO Completed upload id=526297lb # [ 168.009329] niks3-server[1138]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6298worker1 # [ 167.998863] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16299worker1 # [ 168.020644] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:13Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06300worker1 # [ 168.025307] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:13Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16301worker1 # [ 168.030538] nix-daemon[834]: reaped child process 1725, status = succeeded6302worker1 # [ 168.059546] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826303worker1 # [ 168.081443] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826304worker1 # [ 168.095839] nix-grpc-daemon[1602]: time=2026-09-22T11:01:13.690Z level=INFO msg="All 2 paths already cached"6305client # copying 1 paths to 'lb:50051'...6306worker1 # [ 168.106298] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826307client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6308worker1 # [ 168.129431] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6309worker1 # [ 168.294593] nix-daemon[834]: accepted connection from pid 1752, user nix-grpc-daemon (trusted)6310worker1 # [ 168.321478] nix-daemon[834]: reaped child process 1755, status = succeeded6311worker1 # [ 168.419574] nix-daemon[834]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6312worker1 # [ 168.436102] nix-daemon[834]: reaped child process 1760, status = succeeded6313lb # [ 168.464917] niks3-server[1138]: 2026/09/22 11:01:14 INFO Received uploads request method=POST path=/api/pending_closures6314worker1 # [ 168.452233] nix-grpc-daemon[1602]: time=2026-09-22T11:01:14.047Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6315worker1 # [ 168.455172] nix-grpc-daemon[1602]: time=2026-09-22T11:01:14.047Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6316lb # [ 168.493380] niks3-server[1138]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6317worker1 # [ 168.485579] nix-grpc-daemon[1602]: time=2026-09-22T11:01:14.080Z level=INFO msg="Uploading 1 narinfos"6318lb # [ 168.508081] niks3-server[1138]: 2026/09/22 11:01:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6319lb # [ 168.519428] niks3-server[1138]: 2026/09/22 11:01:14 INFO Signed narinfos id=53 count=16320lb # [ 168.526477] niks3-server[1138]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6321worker1 # [ 168.516969] nix-grpc-daemon[1602]: time=2026-09-22T11:01:14.112Z level=INFO msg="Upload complete. (369ms)"6322lb # [ 168.538278] niks3-server[1138]: 2026/09/22 11:01:14 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6323lb # [ 168.544219] niks3-server[1138]: 2026/09/22 11:01:14 INFO Completed upload id=536324worker1 # [ 168.524094] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=7686325lb # [ 168.550366] niks3-server[1138]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6326worker1 # [ 168.535754] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826327worker1 # [ 168.542089] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6328worker1 # [ 168.546622] nix-daemon[834]: reaped child process 1750, status = succeeded6329client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6330worker1 # [ 168.559471] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6331client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6332client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6333client # farm-top> LOG-farm-top-during-drain6334worker1 # [ 169.463772] nix-daemon[834]: reaped child process 1764, status = succeeded6335worker1 # [ 169.534279] nix-daemon[834]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6336worker1 # [ 169.557789] nix-daemon[834]: reaped child process 1783, status = succeeded6337lb # [ 169.578583] niks3-server[1138]: 2026/09/22 11:01:15 INFO Received uploads request method=POST path=/api/pending_closures6338worker1 # [ 169.570216] nix-grpc-daemon[1602]: time=2026-09-22T11:01:15.163Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6339worker1 # [ 169.571765] nix-grpc-daemon[1602]: time=2026-09-22T11:01:15.163Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6340lb # [ 169.619727] niks3-server[1138]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6341lb # [ 169.633374] niks3-server[1138]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6342lb # [ 169.643922] niks3-server[1138]: 2026/09/22 11:01:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6343worker1 # [ 169.628531] nix-grpc-daemon[1602]: time=2026-09-22T11:01:15.223Z level=INFO msg="Uploading 1 narinfos"6344lb # [ 169.653205] niks3-server[1138]: 2026/09/22 11:01:15 INFO Signed narinfos id=54 count=16345lb # [ 169.659510] niks3-server[1138]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6346lb # [ 169.668596] niks3-server[1138]: 2026/09/22 11:01:15 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6347lb # [ 169.676558] niks3-server[1138]: 2026/09/22 11:01:15 INFO Completed upload id=546348worker1 # [ 169.655458] nix-grpc-daemon[1602]: time=2026-09-22T11:01:15.250Z level=INFO msg="Upload complete. (286ms)"6349lb # [ 169.681832] niks3-server[1138]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6350worker1 # [ 169.668973] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16351worker1 # [ 169.689917] nix-daemon[834]: reaped child process 1762, status = succeeded6352client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.75 seconds)6353client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6354client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.21 seconds)6355worker2: must succeed: pkill -f 'read -t [9]0 x'6356worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6357client: waiting for success: ! systemctl is-active sw6358worker2 # [ 170.260751] nix-daemon[855]: reaped child process 1784, status = succeeded6359worker2 # [ 170.368645] nix-daemon[855]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)6360worker2 # [ 170.389759] nix-daemon[855]: reaped child process 1955, status = succeeded6361lb # [ 170.371775] niks3-server[1138]: 2026/09/22 11:01:15 INFO Received uploads request method=POST path=/api/pending_closures6362worker2 # [ 170.403604] nix-grpc-daemon[1720]: time=2026-09-22T11:01:16.001Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6363worker2 # [ 170.405427] nix-grpc-daemon[1720]: time=2026-09-22T11:01:16.001Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6364lb # [ 170.405164] niks3-server[1138]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6365lb # [ 170.424145] niks3-server[1138]: 2026/09/22 11:01:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6366worker2 # [ 170.453941] nix-grpc-daemon[1720]: time=2026-09-22T11:01:16.052Z level=INFO msg="Uploading 1 narinfos"6367lb # [ 170.440655] niks3-server[1138]: 2026/09/22 11:01:16 INFO Signed narinfos id=55 count=16368lb # [ 170.443502] niks3-server[1138]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6369lb # [ 170.457239] niks3-server[1138]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6370worker2 # [ 170.486364] nix-grpc-daemon[1720]: time=2026-09-22T11:01:16.085Z level=INFO msg="Upload complete. (290ms)"6371lb # [ 170.466715] niks3-server[1138]: 2026/09/22 11:01:16 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6372lb # [ 170.474565] niks3-server[1138]: 2026/09/22 11:01:16 INFO Completed upload id=556373lb # [ 170.478502] niks3-server[1138]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6374worker2 # [ 170.501243] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43404 duration_s=14 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16375worker2 # [ 170.519578] nix-daemon[855]: reaped child process 1782, status = succeeded6376client # [ 170.496447] systemd[1]: sw.service: Deactivated successfully.6377client # [ 170.501968] systemd[1]: sw.service: Consumed 196ms CPU time over 16.128s wall clock time, 43.4M memory peak, 7.3K incoming IP traffic, 8.3K outgoing IP traffic.6378worker2 # [ 171.039243] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:16Z level=info event=signal_exit6379client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6380client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6381client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6382worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6383worker2 # [ 171.261149] systemd[1]: Starting scheduler-token.service...6384worker2 # [ 171.335611] systemd[1]: scheduler-token.service: Deactivated successfully.6385worker2 # [ 171.341006] systemd[1]: Finished scheduler-token.service.6386worker2 # [ 171.342633] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6387worker1 # [ 173.003152] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:520426388worker2 # [ 173.047519] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:18Z level=info event=scheduler_restarting6389worker1 # [ 173.007905] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:18Z level=info event=worker_gone addr=192.168.1.4:500516390worker2 # [ 173.069653] nix-grpc-daemon[1639]: ts=2026-09-22T11:01:18Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6391worker2 # [ 173.095366] nix-daemon[855]: reaped child process 1691, status = succeeded6392worker2 # [ 173.102264] nix-grpc-daemon[1639]: E0000 00:00:1790074878.701263 1684 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6393worker2 # [ 173.109044] nix-grpc-daemon[1639]: E0000 00:00:1790074878.701514 1684 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6394worker2 # [ 174.059613] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6395worker2 # [ 174.059823] systemd[1]: nix-grpc-daemon.service: Consumed 844ms CPU time over 30.304s wall clock time, 35.3M memory peak, 21.5K incoming IP traffic, 35.3K outgoing IP traffic.6396worker2 # [ 174.072276] systemd[1]: Starting Nix worker-protocol over gRPC...6397worker2 # [ 174.198038] nix-grpc-daemon[1972]: ts=2026-09-22T11:01:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6398worker2 # [ 174.200757] nix-grpc-daemon[1972]: ts=2026-09-22T11:01:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6399worker2 # [ 174.205027] nix-grpc-daemon[1972]: ts=2026-09-22T11:01:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16400worker2 # [ 174.212316] nix-grpc-daemon[1972]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6401worker2 # [ 174.244232] nix-grpc-daemon[1972]: E0000 00:00:1790074879.809231 1972 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6402worker2 # [ 174.246761] nix-grpc-daemon[1972]: E0000 00:00:1790074879.815348 1972 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6403worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.09 seconds)6404lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6405lb: must succeed: curl -sf localhost:9901/clusters6406worker2 # [ 174.269223] nix-grpc-daemon[1972]: ts=2026-09-22T11:01:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6407worker2 # [ 174.274466] nix-grpc-daemon[1972]: E0000 00:00:1790074879.823304 2011 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6408worker2 # [ 174.279799] nix-grpc-daemon[1972]: E0000 00:00:1790074879.843240 2011 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6409worker2 # [ 174.282680] systemd[1]: Started Nix worker-protocol over gRPC.6410worker1 # [ 174.239468] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:520426411worker2 # [ 174.283521] nix-grpc-daemon[1972]: E0000 00:00:1790074879.848761 2011 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6412worker2 # [ 174.286097] nix-grpc-daemon[1972]: E0000 00:00:1790074879.865201 2011 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6413worker2 # [ 174.288418] nix-grpc-daemon[1972]: ts=2026-09-22T11:01:19Z level=info event=scheduler_connected addr=lb:500516414worker1 # [ 174.243757] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06415lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6416lb: must succeed: curl -sf localhost:9901/clusters6417lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6418lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.11 seconds)6419lb: must succeed: curl -sf localhost:9901/clusters6420lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6421worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6422worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6423(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 21.14 seconds)6424subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6425client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16426worker1 # [ 175.682852] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:348826427worker1 # [ 175.693534] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826428worker1 # [ 175.741877] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06429worker1 # [ 175.749327] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16430worker1 # [ 175.755553] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06431worker1 # [ 175.761161] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06432worker1 # [ 175.787904] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826433worker1 # [ 175.804361] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826434worker1 # [ 175.818995] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826435worker1 # [ 175.834631] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6436worker1 # [ 175.987302] nix-daemon[834]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6437worker1 # [ 176.017528] nix-daemon[834]: reaped child process 1795, status = succeeded6438worker1 # [ 176.117533] nix-daemon[834]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6439worker1 # [ 176.139176] nix-daemon[834]: reaped child process 1800, status = succeeded6440lb # [ 176.159071] niks3-server[1138]: 2026/09/22 11:01:21 INFO Received uploads request method=POST path=/api/pending_closures6441worker1 # [ 176.148506] nix-grpc-daemon[1602]: time=2026-09-22T11:01:21.743Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6442worker1 # [ 176.150018] nix-grpc-daemon[1602]: time=2026-09-22T11:01:21.743Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6443lb # [ 176.192470] niks3-server[1138]: 2026/09/22 11:01:21 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6444worker1 # [ 176.187075] nix-grpc-daemon[1602]: time=2026-09-22T11:01:21.782Z level=INFO msg="Uploading 1 narinfos"6445lb # [ 176.208205] niks3-server[1138]: 2026/09/22 11:01:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6446lb # [ 176.219519] niks3-server[1138]: 2026/09/22 11:01:21 INFO Signed narinfos id=56 count=16447lb # [ 176.227765] niks3-server[1138]: 2026/09/22 11:01:21 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6448lb # [ 176.242234] niks3-server[1138]: 2026/09/22 11:01:21 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6449worker1 # [ 176.225464] nix-grpc-daemon[1602]: time=2026-09-22T11:01:21.820Z level=INFO msg="Upload complete. (375ms)"6450lb # [ 176.250739] niks3-server[1138]: 2026/09/22 11:01:21 INFO Completed upload id=566451lb # [ 176.254088] niks3-server[1138]: 2026/09/22 11:01:21 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6452worker1 # [ 176.236326] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=5126453worker1 # [ 176.244414] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826454worker1 # [ 176.257824] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6455worker1 # [ 176.263171] nix-daemon[834]: reaped child process 1790, status = succeeded6456worker1 # [ 176.277398] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6457worker1 # [ 177.986633] nix-daemon[834]: reaped child process 1804, status = succeeded6458worker1 # [ 178.061788] nix-daemon[834]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6459lb # [ 178.104433] niks3-server[1138]: 2026/09/22 11:01:23 INFO Received uploads request method=POST path=/api/pending_closures6460worker1 # [ 178.086571] nix-daemon[834]: reaped child process 1823, status = succeeded6461worker1 # [ 178.091807] nix-grpc-daemon[1602]: time=2026-09-22T11:01:23.684Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6462worker1 # [ 178.093762] nix-grpc-daemon[1602]: time=2026-09-22T11:01:23.684Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6463lb # [ 178.131751] niks3-server[1138]: 2026/09/22 11:01:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6464worker1 # [ 178.118090] nix-grpc-daemon[1602]: time=2026-09-22T11:01:23.713Z level=INFO msg="Uploading 1 narinfos"6465lb # [ 178.141378] niks3-server[1138]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6466lb # [ 178.152182] niks3-server[1138]: 2026/09/22 11:01:23 INFO Signed narinfos id=57 count=16467lb # [ 178.156365] niks3-server[1138]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6468worker1 # [ 178.143235] nix-grpc-daemon[1602]: time=2026-09-22T11:01:23.738Z level=INFO msg="Upload complete. (253ms)"6469lb # [ 178.166525] niks3-server[1138]: 2026/09/22 11:01:23 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6470worker1 # [ 178.151585] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16471lb # [ 178.175894] niks3-server[1138]: 2026/09/22 11:01:23 INFO Completed upload id=576472lb # [ 178.177891] niks3-server[1138]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6473worker1 # [ 178.160349] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:23Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16474worker1 # [ 178.168755] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826475worker1 # [ 178.170662] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:23Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06476worker1 # [ 178.174212] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826477worker1 # [ 178.177602] nix-daemon[834]: reaped child process 1802, status = succeeded6478worker1 # [ 178.180566] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826479worker1 # [ 178.187629] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6480worker1 # [ 178.316373] nix-daemon[834]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6481worker1 # [ 178.340811] nix-daemon[834]: reaped child process 1832, status = succeeded6482worker1 # [ 178.459966] nix-daemon[834]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6483worker1 # [ 178.482971] nix-daemon[834]: reaped child process 1837, status = succeeded6484lb # [ 178.510975] niks3-server[1138]: 2026/09/22 11:01:24 INFO Received uploads request method=POST path=/api/pending_closures6485worker1 # [ 178.499509] nix-grpc-daemon[1602]: time=2026-09-22T11:01:24.094Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6486worker1 # [ 178.501339] nix-grpc-daemon[1602]: time=2026-09-22T11:01:24.094Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6487worker1 # [ 178.526156] nix-grpc-daemon[1602]: time=2026-09-22T11:01:24.121Z level=INFO msg="Uploading 1 narinfos"6488lb # [ 178.546882] niks3-server[1138]: 2026/09/22 11:01:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6489lb # [ 178.557847] niks3-server[1138]: 2026/09/22 11:01:24 INFO Signed narinfos id=58 count=16490lb # [ 178.563413] niks3-server[1138]: 2026/09/22 11:01:24 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6491lb # [ 178.576898] niks3-server[1138]: 2026/09/22 11:01:24 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6492lb # [ 178.579656] niks3-server[1138]: 2026/09/22 11:01:24 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6493lb # [ 178.591015] niks3-server[1138]: 2026/09/22 11:01:24 INFO Completed upload id=586494worker1 # [ 178.570235] nix-grpc-daemon[1602]: time=2026-09-22T11:01:24.165Z level=INFO msg="Upload complete. (374ms)"6495lb # [ 178.594403] niks3-server[1138]: 2026/09/22 11:01:24 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6496worker1 # [ 178.576400] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=5126497worker1 # [ 178.592500] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826498worker1 # [ 178.598154] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6499worker1 # [ 178.601882] nix-daemon[834]: reaped child process 1827, status = succeeded6500worker1 # [ 178.610909] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6501lb # [ 178.796611] envoy[898]: conn peer=[2001:db8:1::1]:44244 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9582 tx=4084 ms=54786502lb # [ 178.799892] envoy[898]: conn peer=[2001:db8:1::1]:42424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1455 ms=159136503lb # [ 178.803128] envoy[898]: conn peer=[2001:db8:1::4]:54032 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7565 tx=616 ms=289776504worker1 # [ 179.240866] nix-daemon[834]: reaped child process 1841, status = succeeded6505worker1 # [ 179.349927] nix-daemon[834]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6506lb # [ 179.393559] niks3-server[1138]: 2026/09/22 11:01:25 INFO Received uploads request method=POST path=/api/pending_closures6507worker1 # [ 179.376062] nix-daemon[834]: reaped child process 1860, status = succeeded6508worker1 # [ 179.382067] nix-grpc-daemon[1602]: time=2026-09-22T11:01:24.973Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6509worker1 # [ 179.383593] nix-grpc-daemon[1602]: time=2026-09-22T11:01:24.973Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6510lb # [ 179.420104] niks3-server[1138]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6511lb # [ 179.427413] niks3-server[1138]: 2026/09/22 11:01:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6512lb # [ 179.432313] niks3-server[1138]: 2026/09/22 11:01:25 INFO Signed narinfos id=59 count=16513worker1 # [ 179.411523] nix-grpc-daemon[1602]: time=2026-09-22T11:01:25.006Z level=INFO msg="Uploading 1 narinfos"6514lb # [ 179.441151] niks3-server[1138]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6515lb # [ 179.447031] niks3-server[1138]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6516lb # [ 179.455989] niks3-server[1138]: 2026/09/22 11:01:25 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6517lb # [ 179.461176] niks3-server[1138]: 2026/09/22 11:01:25 INFO Completed upload id=596518worker1 # [ 179.440487] nix-grpc-daemon[1602]: time=2026-09-22T11:01:25.035Z level=INFO msg="Upload complete. (218ms)"6519lb # [ 179.465690] niks3-server[1138]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6520worker1 # [ 179.448712] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16521worker1 # [ 179.457515] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:25Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16522worker1 # [ 179.467077] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826523worker1 # [ 179.470066] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:25Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06524worker1 # [ 179.474329] nix-daemon[834]: reaped child process 1839, status = succeeded6525worker1 # [ 179.500232] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826526worker1 # [ 179.513982] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826527worker1 # [ 179.521334] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6528worker1 # [ 179.674599] nix-daemon[834]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)6529worker1 # [ 179.701595] nix-daemon[834]: reaped child process 1869, status = succeeded6530worker1 # [ 179.825372] nix-daemon[834]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6531lb # [ 179.876956] niks3-server[1138]: 2026/09/22 11:01:25 INFO Received uploads request method=POST path=/api/pending_closures6532worker1 # [ 179.863823] nix-daemon[834]: reaped child process 1874, status = succeeded6533worker1 # [ 179.867318] nix-grpc-daemon[1602]: time=2026-09-22T11:01:25.461Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6534worker1 # [ 179.869034] nix-grpc-daemon[1602]: time=2026-09-22T11:01:25.461Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6535lb # [ 179.907690] niks3-server[1138]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6536worker1 # [ 179.897669] nix-grpc-daemon[1602]: time=2026-09-22T11:01:25.492Z level=INFO msg="Uploading 1 narinfos"6537lb # [ 179.918187] niks3-server[1138]: 2026/09/22 11:01:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6538lb # [ 179.923717] niks3-server[1138]: 2026/09/22 11:01:25 INFO Signed narinfos id=60 count=16539lb # [ 179.927560] niks3-server[1138]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6540lb # [ 179.939162] niks3-server[1138]: 2026/09/22 11:01:25 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6541lb # [ 179.946149] niks3-server[1138]: 2026/09/22 11:01:25 INFO Completed upload id=606542worker1 # [ 179.925571] nix-grpc-daemon[1602]: time=2026-09-22T11:01:25.520Z level=INFO msg="Upload complete. (393ms)"6543lb # [ 179.949136] niks3-server[1138]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6544worker1 # [ 179.932159] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=5126545worker1 # [ 179.935298] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826546worker1 # [ 179.944480] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6547worker1 # [ 179.948113] nix-daemon[834]: reaped child process 1864, status = succeeded6548worker1 # [ 179.959456] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6549worker1 # [ 181.401818] nix-daemon[834]: reaped child process 1878, status = succeeded6550worker1 # [ 181.509385] nix-daemon[834]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6551worker1 # [ 181.533381] nix-daemon[834]: reaped child process 1897, status = succeeded6552lb # [ 181.557763] niks3-server[1138]: 2026/09/22 11:01:27 INFO Received uploads request method=POST path=/api/pending_closures6553worker1 # [ 181.548409] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.142Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6554worker1 # [ 181.550059] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.142Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6555lb # [ 181.596932] niks3-server[1138]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6556lb # [ 181.610916] niks3-server[1138]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6557lb # [ 181.622616] niks3-server[1138]: 2026/09/22 11:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6558worker1 # [ 181.608240] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.203Z level=INFO msg="Uploading 1 narinfos"6559lb # [ 181.632894] niks3-server[1138]: 2026/09/22 11:01:27 INFO Signed narinfos id=61 count=16560lb # [ 181.638519] niks3-server[1138]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6561lb # [ 181.649267] niks3-server[1138]: 2026/09/22 11:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6562lb # [ 181.657596] niks3-server[1138]: 2026/09/22 11:01:27 INFO Completed upload id=616563worker1 # [ 181.636810] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.231Z level=INFO msg="Upload complete. (267ms)"6564lb # [ 181.662139] niks3-server[1138]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6565worker1 # [ 181.654320] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16566worker1 # [ 181.672333] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:27Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16567worker1 # [ 181.684604] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826568worker1 # [ 181.689254] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:27Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06569worker1 # [ 181.700157] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826570worker1 # [ 181.703214] nix-daemon[834]: reaped child process 1876, status = succeeded6571worker1 # [ 181.711010] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6572worker1 # [ 181.715519] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.283Z level=INFO msg="All 1 paths already cached"6573worker1 # [ 181.721732] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826574worker1 # [ 181.849148] nix-daemon[834]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)6575worker1 # [ 181.869283] nix-daemon[834]: reaped child process 1906, status = succeeded6576worker1 # [ 181.978962] nix-daemon[834]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6577worker1 # [ 182.001126] nix-daemon[834]: reaped child process 1911, status = succeeded6578lb # [ 182.019818] niks3-server[1138]: 2026/09/22 11:01:27 INFO Received uploads request method=POST path=/api/pending_closures6579worker1 # [ 182.008212] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.603Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6580worker1 # [ 182.009722] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.603Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6581lb # [ 182.052794] niks3-server[1138]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6582lb # [ 182.069622] niks3-server[1138]: 2026/09/22 11:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6583worker1 # [ 182.056035] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.650Z level=INFO msg="Uploading 1 narinfos"6584lb # [ 182.082834] niks3-server[1138]: 2026/09/22 11:01:27 INFO Signed narinfos id=62 count=16585lb # [ 182.091044] niks3-server[1138]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6586lb # [ 182.102864] niks3-server[1138]: 2026/09/22 11:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6587worker1 # [ 182.089009] nix-grpc-daemon[1602]: time=2026-09-22T11:01:27.684Z level=INFO msg="Upload complete. (374ms)"6588lb # [ 182.112219] niks3-server[1138]: 2026/09/22 11:01:27 INFO Completed upload id=626589lb # [ 182.115738] niks3-server[1138]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6590worker1 # [ 182.100694] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=6246591worker1 # [ 182.115039] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826592worker1 # [ 182.122028] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6593worker1 # [ 182.128764] nix-daemon[834]: reaped child process 1901, status = succeeded6594worker1 # [ 182.140726] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6595worker1 # [ 183.212156] nix-daemon[834]: reaped child process 1915, status = succeeded6596worker1 # [ 183.339139] nix-daemon[834]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6597worker1 # [ 183.363564] nix-daemon[834]: reaped child process 1934, status = succeeded6598lb # [ 183.383368] niks3-server[1138]: 2026/09/22 11:01:28 INFO Received uploads request method=POST path=/api/pending_closures6599worker1 # [ 183.376555] nix-grpc-daemon[1602]: time=2026-09-22T11:01:28.968Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6600worker1 # [ 183.378169] nix-grpc-daemon[1602]: time=2026-09-22T11:01:28.968Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6601lb # [ 183.425513] niks3-server[1138]: 2026/09/22 11:01:29 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6602lb # [ 183.437985] niks3-server[1138]: 2026/09/22 11:01:29 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6603worker1 # [ 183.429474] nix-grpc-daemon[1602]: time=2026-09-22T11:01:29.024Z level=INFO msg="Uploading 1 narinfos"6604lb # [ 183.450462] niks3-server[1138]: 2026/09/22 11:01:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6605lb # [ 183.459476] niks3-server[1138]: 2026/09/22 11:01:29 INFO Signed narinfos id=63 count=16606lb # [ 183.465060] niks3-server[1138]: 2026/09/22 11:01:29 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6607lb # [ 183.474932] niks3-server[1138]: 2026/09/22 11:01:29 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6608worker1 # [ 183.460405] nix-grpc-daemon[1602]: time=2026-09-22T11:01:29.055Z level=INFO msg="Upload complete. (304ms)"6609lb # [ 183.484481] niks3-server[1138]: 2026/09/22 11:01:29 INFO Completed upload id=636610lb # [ 183.490514] niks3-server[1138]: 2026/09/22 11:01:29 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6611worker1 # [ 183.474637] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16612worker1 # [ 183.486022] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:29Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16613worker1 # [ 183.494911] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826614worker1 # [ 183.499836] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:29Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06615worker1 # [ 183.505376] nix-daemon[834]: reaped child process 1913, status = succeeded6616worker1 # [ 183.509420] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826617worker1 # [ 183.523411] nix-grpc-daemon[1602]: time=2026-09-22T11:01:29.118Z level=INFO msg="All 1 paths already cached"6618worker1 # [ 183.529842] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826619worker1 # [ 183.536873] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6620worker1 # [ 183.656886] nix-daemon[834]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6621worker1 # [ 183.682559] nix-daemon[834]: reaped child process 1943, status = succeeded6622worker1 # [ 183.794860] nix-daemon[834]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)6623worker1 # [ 183.818516] nix-daemon[834]: reaped child process 1948, status = succeeded6624lb # [ 183.842252] niks3-server[1138]: 2026/09/22 11:01:29 INFO Received uploads request method=POST path=/api/pending_closures6625worker1 # [ 183.831962] nix-grpc-daemon[1602]: time=2026-09-22T11:01:29.426Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6626worker1 # [ 183.834075] nix-grpc-daemon[1602]: time=2026-09-22T11:01:29.426Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6627lb # [ 183.874192] niks3-server[1138]: 2026/09/22 11:01:29 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6628lb # [ 183.890180] niks3-server[1138]: 2026/09/22 11:01:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6629worker1 # [ 183.875285] nix-grpc-daemon[1602]: time=2026-09-22T11:01:29.470Z level=INFO msg="Uploading 1 narinfos"6630lb # [ 183.904986] niks3-server[1138]: 2026/09/22 11:01:29 INFO Signed narinfos id=64 count=16631lb # [ 183.911143] niks3-server[1138]: 2026/09/22 11:01:29 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6632lb # [ 183.925736] niks3-server[1138]: 2026/09/22 11:01:29 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6633lb # [ 183.938311] niks3-server[1138]: 2026/09/22 11:01:29 INFO Completed upload id=646634worker1 # [ 183.917608] nix-grpc-daemon[1602]: time=2026-09-22T11:01:29.512Z level=INFO msg="Upload complete. (371ms)"6635lb # [ 183.941910] niks3-server[1138]: 2026/09/22 11:01:29 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6636worker1 # [ 183.928402] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=6246637worker1 # [ 183.944552] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826638worker1 # [ 183.953674] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6639worker1 # [ 183.960475] nix-daemon[834]: reaped child process 1938, status = succeeded6640worker1 # [ 183.977602] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6641worker1 # [ 185.046309] nix-daemon[834]: reaped child process 1952, status = succeeded6642worker1 # [ 185.110404] nix-daemon[834]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)6643worker1 # [ 185.133926] nix-daemon[834]: reaped child process 1971, status = succeeded6644lb # [ 185.154007] niks3-server[1138]: 2026/09/22 11:01:30 INFO Received uploads request method=POST path=/api/pending_closures6645worker1 # [ 185.148347] nix-grpc-daemon[1602]: time=2026-09-22T11:01:30.742Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6646worker1 # [ 185.150366] nix-grpc-daemon[1602]: time=2026-09-22T11:01:30.742Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6647lb # [ 185.180979] niks3-server[1138]: 2026/09/22 11:01:30 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6648lb # [ 185.192822] niks3-server[1138]: 2026/09/22 11:01:30 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6649worker1 # [ 185.176950] nix-grpc-daemon[1602]: time=2026-09-22T11:01:30.771Z level=INFO msg="Uploading 1 narinfos"6650lb # [ 185.198735] niks3-server[1138]: 2026/09/22 11:01:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6651lb # [ 185.202762] niks3-server[1138]: 2026/09/22 11:01:30 INFO Signed narinfos id=65 count=16652lb # [ 185.207854] niks3-server[1138]: 2026/09/22 11:01:30 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6653lb # [ 185.212542] niks3-server[1138]: 2026/09/22 11:01:30 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6654lb # [ 185.218441] niks3-server[1138]: 2026/09/22 11:01:30 INFO Completed upload id=656655worker1 # [ 185.195886] nix-grpc-daemon[1602]: time=2026-09-22T11:01:30.791Z level=INFO msg="Upload complete. (206ms)"6656lb # [ 185.219674] niks3-server[1138]: 2026/09/22 11:01:30 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6657worker1 # [ 185.204837] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16658worker1 # [ 185.217076] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:30Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16659worker1 # [ 185.220109] nix-daemon[834]: reaped child process 1950, status = succeeded6660worker1 # [ 185.247546] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826661worker1 # [ 185.253108] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:30Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06662worker1 # [ 185.260507] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826663worker1 # [ 185.269210] nix-grpc-daemon[1602]: time=2026-09-22T11:01:30.864Z level=INFO msg="All 1 paths already cached"6664worker1 # [ 185.277017] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826665worker1 # [ 185.283523] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6666worker1 # [ 185.402864] nix-daemon[834]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)6667worker1 # [ 185.425906] nix-daemon[834]: reaped child process 1980, status = succeeded6668worker1 # [ 185.549282] nix-daemon[834]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6669lb # [ 185.592766] niks3-server[1138]: 2026/09/22 11:01:31 INFO Received uploads request method=POST path=/api/pending_closures6670worker1 # [ 185.574281] nix-daemon[834]: reaped child process 1985, status = succeeded6671worker1 # [ 185.579462] nix-grpc-daemon[1602]: time=2026-09-22T11:01:31.174Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6672worker1 # [ 185.581427] nix-grpc-daemon[1602]: time=2026-09-22T11:01:31.174Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6673lb # [ 185.622037] niks3-server[1138]: 2026/09/22 11:01:31 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6674lb # [ 185.629346] niks3-server[1138]: 2026/09/22 11:01:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6675lb # [ 185.634065] niks3-server[1138]: 2026/09/22 11:01:31 INFO Signed narinfos id=66 count=16676worker1 # [ 185.613398] nix-grpc-daemon[1602]: time=2026-09-22T11:01:31.208Z level=INFO msg="Uploading 1 narinfos"6677lb # [ 185.636943] niks3-server[1138]: 2026/09/22 11:01:31 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6678lb # [ 185.647195] niks3-server[1138]: 2026/09/22 11:01:31 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6679lb # [ 185.652874] niks3-server[1138]: 2026/09/22 11:01:31 INFO Completed upload id=666680worker1 # [ 185.632309] nix-grpc-daemon[1602]: time=2026-09-22T11:01:31.227Z level=INFO msg="Upload complete. (341ms)"6681lb # [ 185.659038] niks3-server[1138]: 2026/09/22 11:01:31 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6682worker1 # [ 185.638185] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=6246683worker1 # [ 185.642998] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826684worker1 # [ 185.655791] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6685worker1 # [ 185.660212] nix-daemon[834]: reaped child process 1975, status = succeeded6686worker1 # [ 185.669387] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6687worker1 # [ 186.753111] nix-daemon[834]: reaped child process 1989, status = succeeded6688worker1 # [ 186.851245] nix-daemon[834]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)6689worker1 # [ 186.872418] nix-daemon[834]: reaped child process 2008, status = succeeded6690lb # [ 186.897773] niks3-server[1138]: 2026/09/22 11:01:32 INFO Received uploads request method=POST path=/api/pending_closures6691worker1 # [ 186.890290] nix-grpc-daemon[1602]: time=2026-09-22T11:01:32.483Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6692worker1 # [ 186.891779] nix-grpc-daemon[1602]: time=2026-09-22T11:01:32.483Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6693lb # [ 186.936299] niks3-server[1138]: 2026/09/22 11:01:32 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6694lb # [ 186.951200] niks3-server[1138]: 2026/09/22 11:01:32 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6695worker1 # [ 186.945339] nix-grpc-daemon[1602]: time=2026-09-22T11:01:32.540Z level=INFO msg="Uploading 1 narinfos"6696lb # [ 186.966722] niks3-server[1138]: 2026/09/22 11:01:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6697lb # [ 186.972663] niks3-server[1138]: 2026/09/22 11:01:32 INFO Signed narinfos id=67 count=16698lb # [ 186.979195] niks3-server[1138]: 2026/09/22 11:01:32 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6699lb # [ 186.991821] niks3-server[1138]: 2026/09/22 11:01:32 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6700worker1 # [ 186.976610] nix-grpc-daemon[1602]: time=2026-09-22T11:01:32.571Z level=INFO msg="Upload complete. (295ms)"6701lb # [ 186.999442] niks3-server[1138]: 2026/09/22 11:01:32 INFO Completed upload id=676702worker1 # [ 186.993431] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16703worker1 # [ 187.008433] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:32Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16704worker1 # [ 187.014254] nix-daemon[834]: reaped child process 1987, status = succeeded6705worker1 # [ 187.030794] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826706worker1 # [ 187.039471] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:32Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06707worker1 # [ 187.050981] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826708worker1 # [ 187.066509] nix-grpc-daemon[1602]: time=2026-09-22T11:01:32.661Z level=INFO msg="All 2 paths already cached"6709worker1 # [ 187.076118] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826710worker1 # [ 187.090176] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6711worker1 # [ 187.239973] nix-daemon[834]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6712worker1 # [ 187.267375] nix-daemon[834]: reaped child process 2017, status = succeeded6713worker1 # [ 187.371634] nix-daemon[834]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6714worker1 # [ 187.393736] nix-daemon[834]: reaped child process 2022, status = succeeded6715lb # [ 187.416269] niks3-server[1138]: 2026/09/22 11:01:33 INFO Received uploads request method=POST path=/api/pending_closures6716worker1 # [ 187.406181] nix-grpc-daemon[1602]: time=2026-09-22T11:01:33.001Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6717worker1 # [ 187.407905] nix-grpc-daemon[1602]: time=2026-09-22T11:01:33.001Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6718lb # [ 187.449122] niks3-server[1138]: 2026/09/22 11:01:33 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6719worker1 # [ 187.446339] nix-grpc-daemon[1602]: time=2026-09-22T11:01:33.041Z level=INFO msg="Uploading 1 narinfos"6720lb # [ 187.468091] niks3-server[1138]: 2026/09/22 11:01:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6721lb # [ 187.478189] niks3-server[1138]: 2026/09/22 11:01:33 INFO Signed narinfos id=68 count=16722lb # [ 187.485752] niks3-server[1138]: 2026/09/22 11:01:33 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6723lb # [ 187.497992] niks3-server[1138]: 2026/09/22 11:01:33 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6724lb # [ 187.505503] niks3-server[1138]: 2026/09/22 11:01:33 INFO Completed upload id=686725worker1 # [ 187.483528] nix-grpc-daemon[1602]: time=2026-09-22T11:01:33.078Z level=INFO msg="Upload complete. (378ms)"6726lb # [ 187.509729] niks3-server[1138]: 2026/09/22 11:01:33 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6727worker1 # [ 187.492360] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=7526728worker1 # [ 187.508585] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826729worker1 # [ 187.515189] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6730worker1 # [ 187.521962] nix-daemon[834]: reaped child process 2012, status = succeeded6731worker1 # [ 187.534322] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6732worker1 # [ 188.644431] nix-daemon[834]: reaped child process 2026, status = succeeded6733worker1 # [ 188.719026] nix-daemon[834]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)6734worker1 # [ 188.742746] nix-daemon[834]: reaped child process 2045, status = succeeded6735lb # [ 188.769522] niks3-server[1138]: 2026/09/22 11:01:34 INFO Received uploads request method=POST path=/api/pending_closures6736worker1 # [ 188.764478] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.358Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6737worker1 # [ 188.766252] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.358Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6738lb # [ 188.806258] niks3-server[1138]: 2026/09/22 11:01:34 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6739lb # [ 188.826879] niks3-server[1138]: 2026/09/22 11:01:34 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6740lb # [ 188.840570] niks3-server[1138]: 2026/09/22 11:01:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6741worker1 # [ 188.824127] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.418Z level=INFO msg="Uploading 1 narinfos"6742lb # [ 188.847449] niks3-server[1138]: 2026/09/22 11:01:34 INFO Signed narinfos id=69 count=16743lb # [ 188.854291] niks3-server[1138]: 2026/09/22 11:01:34 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6744lb # [ 188.863619] niks3-server[1138]: 2026/09/22 11:01:34 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6745lb # [ 188.870889] niks3-server[1138]: 2026/09/22 11:01:34 INFO Completed upload id=696746worker1 # [ 188.847286] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.442Z level=INFO msg="Upload complete. (302ms)"6747lb # [ 188.873019] niks3-server[1138]: 2026/09/22 11:01:34 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6748worker1 # [ 188.865248] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16749worker1 # [ 188.878418] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:34Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16750worker1 # [ 188.882797] nix-daemon[834]: reaped child process 2024, status = succeeded6751worker1 # [ 188.903855] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826752worker1 # [ 188.918144] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826753worker1 # [ 188.934091] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.529Z level=INFO msg="All 2 paths already cached"6754worker1 # [ 188.943435] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826755worker1 # [ 188.958138] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6756worker1 # [ 189.107839] nix-daemon[834]: accepted connection from pid 2049, user nix-grpc-daemon (trusted)6757worker1 # [ 189.125775] nix-daemon[834]: reaped child process 2052, status = succeeded6758worker1 # [ 189.238345] nix-daemon[834]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)6759lb # [ 189.276979] niks3-server[1138]: 2026/09/22 11:01:34 INFO Received uploads request method=POST path=/api/pending_closures6760worker1 # [ 189.260292] nix-daemon[834]: reaped child process 2057, status = succeeded6761worker1 # [ 189.283442] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.878Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6762worker1 # [ 189.285124] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.878Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6763worker1 # [ 189.318628] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.913Z level=INFO msg="Uploading 1 narinfos"6764lb # [ 189.333545] niks3-server[1138]: 2026/09/22 11:01:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6765lb # [ 189.347870] niks3-server[1138]: 2026/09/22 11:01:34 INFO Signed narinfos id=70 count=16766lb # [ 189.355912] niks3-server[1138]: 2026/09/22 11:01:34 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6767worker1 # [ 189.350498] nix-grpc-daemon[1602]: time=2026-09-22T11:01:34.945Z level=INFO msg="Upload complete. (377ms)"6768lb # [ 189.373370] niks3-server[1138]: 2026/09/22 11:01:34 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6769worker1 # [ 189.360355] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=7526770lb # [ 189.382844] niks3-server[1138]: 2026/09/22 11:01:34 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6771lb # [ 189.387592] niks3-server[1138]: 2026/09/22 11:01:34 INFO Completed upload id=706772lb # [ 189.390922] niks3-server[1138]: 2026/09/22 11:01:34 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6773worker1 # [ 189.376344] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826774worker1 # [ 189.384849] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6775worker1 # [ 189.390308] nix-daemon[834]: reaped child process 2047, status = succeeded6776worker1 # [ 189.405837] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6777worker1 # [ 190.275025] nix-daemon[834]: reaped child process 2061, status = succeeded6778worker1 # [ 190.396142] nix-daemon[834]: accepted connection from pid 2077, user nix-grpc-daemon (trusted)6779worker1 # [ 190.417143] nix-daemon[834]: reaped child process 2080, status = succeeded6780lb # [ 190.443054] niks3-server[1138]: 2026/09/22 11:01:36 INFO Received uploads request method=POST path=/api/pending_closures6781worker1 # [ 190.434403] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.027Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6782worker1 # [ 190.436500] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.027Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6783lb # [ 190.485265] niks3-server[1138]: 2026/09/22 11:01:36 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6784lb # [ 190.500125] niks3-server[1138]: 2026/09/22 11:01:36 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6785worker1 # [ 190.489891] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.084Z level=INFO msg="Uploading 1 narinfos"6786lb # [ 190.512398] niks3-server[1138]: 2026/09/22 11:01:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6787lb # [ 190.520960] niks3-server[1138]: 2026/09/22 11:01:36 INFO Signed narinfos id=71 count=16788lb # [ 190.523650] niks3-server[1138]: 2026/09/22 11:01:36 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6789lb # [ 190.533930] niks3-server[1138]: 2026/09/22 11:01:36 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6790lb # [ 190.543061] niks3-server[1138]: 2026/09/22 11:01:36 INFO Completed upload id=716791worker1 # [ 190.521500] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.116Z level=INFO msg="Upload complete. (304ms)"6792lb # [ 190.546163] niks3-server[1138]: 2026/09/22 11:01:36 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6793worker1 # [ 190.537419] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16794worker1 # [ 190.555878] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:36Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16795worker1 # [ 190.561026] nix-daemon[834]: reaped child process 2059, status = succeeded6796worker1 # [ 190.579400] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826797worker1 # [ 190.595815] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826798worker1 # [ 190.611270] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.206Z level=INFO msg="All 2 paths already cached"6799worker1 # [ 190.621461] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:348826800worker1 # [ 190.637216] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6801worker1 # [ 190.797572] nix-daemon[834]: accepted connection from pid 2084, user nix-grpc-daemon (trusted)6802worker1 # [ 190.817200] nix-daemon[834]: reaped child process 2087, status = succeeded6803worker1 # [ 190.933486] nix-daemon[834]: accepted connection from pid 2089, user nix-grpc-daemon (trusted)6804worker1 # [ 190.956825] nix-daemon[834]: reaped child process 2092, status = succeeded6805lb # [ 190.979147] niks3-server[1138]: 2026/09/22 11:01:36 INFO Received uploads request method=POST path=/api/pending_closures6806worker1 # [ 190.967633] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.562Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6807worker1 # [ 190.969408] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.562Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6808lb # [ 191.007774] niks3-server[1138]: 2026/09/22 11:01:36 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6809worker1 # [ 191.008047] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.602Z level=INFO msg="Uploading 1 narinfos"6810lb # [ 191.021858] niks3-server[1138]: 2026/09/22 11:01:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6811lb # [ 191.035439] niks3-server[1138]: 2026/09/22 11:01:36 INFO Signed narinfos id=72 count=16812lb # [ 191.043484] niks3-server[1138]: 2026/09/22 11:01:36 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6813worker1 # [ 191.034113] nix-grpc-daemon[1602]: time=2026-09-22T11:01:36.629Z level=INFO msg="Upload complete. (380ms)"6814lb # [ 191.055976] niks3-server[1138]: 2026/09/22 11:01:36 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6815lb # [ 191.061908] niks3-server[1138]: 2026/09/22 11:01:36 INFO Completed upload id=726816lb # [ 191.066790] niks3-server[1138]: 2026/09/22 11:01:36 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6817worker1 # [ 191.045347] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=0 paths=1 nar_bytes_in=7526818worker1 # [ 191.052331] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:348826819worker1 # [ 191.063860] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6820worker1 # [ 191.069315] nix-daemon[834]: reaped child process 2082, status = succeeded6821worker1 # [ 191.080203] nix-daemon[834]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6822worker1 # [ 192.195300] nix-daemon[834]: reaped child process 2096, status = succeeded6823worker1 # [ 192.268307] nix-daemon[834]: accepted connection from pid 2112, user nix-grpc-daemon (trusted)6824worker1 # [ 192.289629] nix-daemon[834]: reaped child process 2115, status = succeeded6825lb # [ 192.317339] niks3-server[1138]: 2026/09/22 11:01:37 INFO Received uploads request method=POST path=/api/pending_closures6826worker1 # [ 192.312489] nix-grpc-daemon[1602]: time=2026-09-22T11:01:37.906Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6827worker1 # [ 192.314417] nix-grpc-daemon[1602]: time=2026-09-22T11:01:37.906Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6828lb # [ 192.362288] niks3-server[1138]: 2026/09/22 11:01:37 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6829lb # [ 192.377814] niks3-server[1138]: 2026/09/22 11:01:37 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6830worker1 # [ 192.365065] nix-grpc-daemon[1602]: time=2026-09-22T11:01:37.959Z level=INFO msg="Uploading 1 narinfos"6831lb # [ 192.390034] niks3-server[1138]: 2026/09/22 11:01:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6832lb # [ 192.398069] niks3-server[1138]: 2026/09/22 11:01:37 INFO Signed narinfos id=73 count=16833lb # [ 192.404561] niks3-server[1138]: 2026/09/22 11:01:38 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6834lb # [ 192.413877] niks3-server[1138]: 2026/09/22 11:01:38 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6835worker1 # [ 192.398101] nix-grpc-daemon[1602]: time=2026-09-22T11:01:37.993Z level=INFO msg="Upload complete. (304ms)"6836lb # [ 192.421477] niks3-server[1138]: 2026/09/22 11:01:38 INFO Completed upload id=736837lb # [ 192.424634] niks3-server[1138]: 2026/09/22 11:01:38 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6838worker1 # [ 192.411665] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34882 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16839worker1 # [ 192.433601] nix-daemon[834]: reaped child process 2094, status = succeeded6840client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.02 seconds)6841worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6842worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6843client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6844client # [ 192.606667] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6845client # Running as unit: gpu.service; invocation ID: 4e43eb2270f541f5b638954c19343b616846client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6847lb: must succeed: curl -sf localhost:9901/clusters6848lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6849lb: must succeed: curl -sf localhost:9901/clusters6850lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6851worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6852worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6853worker2 # [ 192.863546] nix-grpc-daemon[1972]: ts=2026-09-22T11:01:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333646854worker2 # [ 192.878277] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6855client # [ 192.851797] nix[1511]: don't know how to build these paths:6856client # [ 192.853519] nix[1511]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6857worker1 # [ 192.861067] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:348826858worker1 # [ 192.885013] nix-grpc-daemon[1502]: ts=2026-09-22T11:01:38Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06859client # [ 192.884354] nix[1511]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6860lb: must succeed: curl -sf localhost:9901/clusters6861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6862lb: must succeed: curl -sf localhost:9901/clusters6863lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6864worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6865worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6866client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6867lb # [ 198.800932] envoy[898]: conn peer=[2001:db8:1::1]:39410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27127 tx=11375 ms=167436868client # [ 222.887236] nix[1511]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6869client # [ 222.900486] nix[1511]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6870client # [ 222.947007] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6871client # [ 222.954154] systemd[1]: gpu.service: Failed with result 'exit-code'.6872client # [ 222.959045] systemd[1]: gpu.service: Consumed 209ms CPU time over 30.339s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6873client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.13 seconds)6874client: must succeed: journalctl -u gpu -o cat6875client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6876lb: must succeed: curl -sf localhost:9901/clusters6877lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6878lb: must succeed: curl -sf localhost:9901/clusters6879lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6880worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6881worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6882(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.88 seconds)6883subtest: low disk drains a worker and builds go to the other6884worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6885worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.18 seconds)6886worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6887worker1 # [ 224.856346] nix-grpc-daemon[1502]: ts=2026-09-22T11:02:10Z level=info event=unhealthy reason=min_free6888worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)6889client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26890worker2 # [ 225.887441] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333646891client # don't know how to build these paths:6892client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6893worker2 # [ 225.898043] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333646894worker2 # [ 225.914733] nix-grpc-daemon[1972]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 1/5)6895worker2 # [ 225.968400] nix-grpc-daemon[1972]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 2/5)6896worker2 # [ 226.049538] nix-grpc-daemon[1972]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5)6897worker2 # [ 226.168661] nix-grpc-daemon[1972]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 548 ms (attempt 4/5)6898worker2 # [ 226.719249] nix-grpc-daemon[1972]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6899worker2 # [ 226.727155] nix-grpc-daemon[1972]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6900worker1 # [ 226.718213] nix-grpc-daemon[1502]: ts=2026-09-22T11:02:12Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06901worker1 # [ 226.724367] nix-grpc-daemon[1502]: ts=2026-09-22T11:02:12Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16902worker2 # [ 226.808967] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333646903worker2 # [ 226.826277] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333646904client # copying 1 paths to 'lb:50051'...6905client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6906worker2 # [ 226.842431] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333646907worker2 # [ 226.859925] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6908worker2 # [ 226.890003] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:12Z level=info event=niks3_push_started pid=20326909worker2 # [ 227.036853] nix-grpc-daemon[2032]: time=2026-09-22T11:02:12.634Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6910worker2 # [ 227.148114] nix-daemon[855]: accepted connection from pid 2045, user nix-grpc-daemon (trusted)6911worker2 # [ 227.171296] nix-daemon[855]: reaped child process 2048, status = succeeded6912worker2 # [ 227.273935] nix-daemon[855]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6913worker2 # [ 227.293902] nix-daemon[855]: reaped child process 2053, status = succeeded6914lb # [ 227.271825] niks3-server[1138]: 2026/09/22 11:02:12 INFO Received uploads request method=POST path=/api/pending_closures6915worker2 # [ 227.306572] nix-grpc-daemon[2032]: time=2026-09-22T11:02:12.904Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6916worker2 # [ 227.308505] nix-grpc-daemon[2032]: time=2026-09-22T11:02:12.904Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6917lb # [ 227.311086] niks3-server[1138]: 2026/09/22 11:02:12 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6918worker2 # [ 227.347970] nix-grpc-daemon[2032]: time=2026-09-22T11:02:12.946Z level=INFO msg="Uploading 1 narinfos"6919lb # [ 227.325964] niks3-server[1138]: 2026/09/22 11:02:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6920lb # [ 227.336659] niks3-server[1138]: 2026/09/22 11:02:12 INFO Signed narinfos id=74 count=16921lb # [ 227.344731] niks3-server[1138]: 2026/09/22 11:02:12 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6922lb # [ 227.357759] niks3-server[1138]: 2026/09/22 11:02:12 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6923lb # [ 227.368901] niks3-server[1138]: 2026/09/22 11:02:12 INFO Completed upload id=746924worker2 # [ 227.390412] nix-grpc-daemon[2032]: time=2026-09-22T11:02:12.989Z level=INFO msg="Upload complete. (353ms)"6925lb # [ 227.372341] niks3-server[1138]: 2026/09/22 11:02:12 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6926worker2 # [ 227.400257] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33364 duration_s=0 paths=1 nar_bytes_in=5126927worker2 # [ 227.412357] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333646928worker2 # [ 227.424150] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6929worker2 # [ 227.427396] nix-daemon[855]: reaped child process 2030, status = succeeded6930client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6931client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6932worker2 # [ 227.439176] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6933client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6934lb # [ 228.804762] envoy[898]: conn peer=[2001:db8:1::1]:41584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=500 ms=301036935client # farm-a> LOG-farm-a-drain6936worker2 # [ 229.116748] nix-daemon[855]: reaped child process 2057, status = succeeded6937worker2 # [ 229.229964] nix-daemon[855]: accepted connection from pid 2073, user nix-grpc-daemon (trusted)6938worker2 # [ 229.249494] nix-daemon[855]: reaped child process 2076, status = succeeded6939lb # [ 229.234267] niks3-server[1138]: 2026/09/22 11:02:14 INFO Received uploads request method=POST path=/api/pending_closures6940worker2 # [ 229.272488] nix-grpc-daemon[2032]: time=2026-09-22T11:02:14.868Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6941worker2 # [ 229.274388] nix-grpc-daemon[2032]: time=2026-09-22T11:02:14.868Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6942lb # [ 229.275007] niks3-server[1138]: 2026/09/22 11:02:14 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6943lb # [ 229.289008] niks3-server[1138]: 2026/09/22 11:02:14 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6944lb # [ 229.303446] niks3-server[1138]: 2026/09/22 11:02:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6945worker2 # [ 229.332115] nix-grpc-daemon[2032]: time=2026-09-22T11:02:14.930Z level=INFO msg="Uploading 1 narinfos"6946lb # [ 229.314275] niks3-server[1138]: 2026/09/22 11:02:14 INFO Signed narinfos id=75 count=16947lb # [ 229.319828] niks3-server[1138]: 2026/09/22 11:02:14 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6948lb # [ 229.329579] niks3-server[1138]: 2026/09/22 11:02:14 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6949lb # [ 229.339275] niks3-server[1138]: 2026/09/22 11:02:14 INFO Completed upload id=756950worker2 # [ 229.361299] nix-grpc-daemon[2032]: time=2026-09-22T11:02:14.960Z level=INFO msg="Upload complete. (307ms)"6951lb # [ 229.343212] niks3-server[1138]: 2026/09/22 11:02:14 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6952worker2 # [ 229.373196] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33364 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16953worker1 # [ 229.338356] nix-grpc-daemon[1502]: ts=2026-09-22T11:02:14Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06954worker1 # [ 229.344539] nix-grpc-daemon[1502]: ts=2026-09-22T11:02:14Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16955worker2 # [ 229.395075] nix-daemon[855]: reaped child process 2055, status = succeeded6956worker2 # [ 229.426811] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333646957worker2 # [ 229.443301] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333646958worker2 # [ 229.459764] nix-grpc-daemon[2032]: time=2026-09-22T11:02:15.058Z level=INFO msg="All 1 paths already cached"6959client # copying 1 paths to 'lb:50051'...6960client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6961worker2 # [ 229.470143] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333646962worker2 # [ 229.483700] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6963worker2 # [ 229.640833] nix-daemon[855]: accepted connection from pid 2080, user nix-grpc-daemon (trusted)6964worker2 # [ 229.658649] nix-daemon[855]: reaped child process 2083, status = succeeded6965worker2 # [ 229.765773] nix-daemon[855]: accepted connection from pid 2085, user nix-grpc-daemon (trusted)6966worker2 # [ 229.783199] nix-daemon[855]: reaped child process 2088, status = succeeded6967lb # [ 229.767230] niks3-server[1138]: 2026/09/22 11:02:15 INFO Received uploads request method=POST path=/api/pending_closures6968worker2 # [ 229.798621] nix-grpc-daemon[2032]: time=2026-09-22T11:02:15.397Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6969worker2 # [ 229.800225] nix-grpc-daemon[2032]: time=2026-09-22T11:02:15.397Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6970lb # [ 229.796704] niks3-server[1138]: 2026/09/22 11:02:15 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6971worker2 # [ 229.830737] nix-grpc-daemon[2032]: time=2026-09-22T11:02:15.429Z level=INFO msg="Uploading 1 narinfos"6972lb # [ 229.810890] niks3-server[1138]: 2026/09/22 11:02:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6973lb # [ 229.823049] niks3-server[1138]: 2026/09/22 11:02:15 INFO Signed narinfos id=76 count=16974lb # [ 229.829225] niks3-server[1138]: 2026/09/22 11:02:15 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6975lb # [ 229.839286] niks3-server[1138]: 2026/09/22 11:02:15 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6976lb # [ 229.846757] niks3-server[1138]: 2026/09/22 11:02:15 INFO Completed upload id=766977worker2 # [ 229.868303] nix-grpc-daemon[2032]: time=2026-09-22T11:02:15.467Z level=INFO msg="Upload complete. (367ms)"6978lb # [ 229.852151] niks3-server[1138]: 2026/09/22 11:02:15 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6979worker2 # [ 229.876136] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33364 duration_s=0 paths=1 nar_bytes_in=6326980worker2 # [ 229.889722] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333646981worker2 # [ 229.895919] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6982worker2 # [ 229.901421] nix-daemon[855]: reaped child process 2078, status = succeeded6983client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6984worker2 # [ 229.913222] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6985client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6986client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6987client # farm-b> LOG-farm-b-drain6988worker2 # [ 231.623615] nix-daemon[855]: reaped child process 2092, status = succeeded6989worker2 # [ 231.698407] nix-daemon[855]: accepted connection from pid 2108, user nix-grpc-daemon (trusted)6990worker2 # [ 231.719656] nix-daemon[855]: reaped child process 2111, status = succeeded6991lb # [ 231.701642] niks3-server[1138]: 2026/09/22 11:02:17 INFO Received uploads request method=POST path=/api/pending_closures6992worker2 # [ 231.733877] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.332Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6993worker2 # [ 231.735557] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.332Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6994lb # [ 231.742171] niks3-server[1138]: 2026/09/22 11:02:17 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6995lb # [ 231.756360] niks3-server[1138]: 2026/09/22 11:02:17 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6996worker2 # [ 231.788641] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.387Z level=INFO msg="Uploading 1 narinfos"6997lb # [ 231.768748] niks3-server[1138]: 2026/09/22 11:02:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6998lb # [ 231.771926] niks3-server[1138]: 2026/09/22 11:02:17 INFO Signed narinfos id=77 count=16999lb # [ 231.780797] niks3-server[1138]: 2026/09/22 11:02:17 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7000lb # [ 231.787759] niks3-server[1138]: 2026/09/22 11:02:17 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7001worker2 # [ 231.813989] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.413Z level=INFO msg="Upload complete. (287ms)"7002lb # [ 231.794220] niks3-server[1138]: 2026/09/22 11:02:17 INFO Completed upload id=777003lb # [ 231.797510] niks3-server[1138]: 2026/09/22 11:02:17 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7004worker2 # [ 231.825151] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33364 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=17005worker1 # [ 231.790345] nix-grpc-daemon[1502]: ts=2026-09-22T11:02:17Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07006worker1 # [ 231.796570] nix-grpc-daemon[1502]: ts=2026-09-22T11:02:17Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17007worker2 # [ 231.847375] nix-daemon[855]: reaped child process 2090, status = succeeded7008worker2 # [ 231.878381] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333647009worker2 # [ 231.895180] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333647010client # copying 1 paths to 'lb:50051'...7011worker2 # [ 231.909830] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.509Z level=INFO msg="All 2 paths already cached"7012client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7013worker2 # [ 231.916878] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333647014worker2 # [ 231.930609] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)7015worker2 # [ 232.090281] nix-daemon[855]: accepted connection from pid 2115, user nix-grpc-daemon (trusted)7016worker2 # [ 232.110174] nix-daemon[855]: reaped child process 2118, status = succeeded7017worker2 # [ 232.219748] nix-daemon[855]: accepted connection from pid 2120, user nix-grpc-daemon (trusted)7018worker2 # [ 232.239543] nix-daemon[855]: reaped child process 2123, status = succeeded7019lb # [ 232.226220] niks3-server[1138]: 2026/09/22 11:02:17 INFO Received uploads request method=POST path=/api/pending_closures7020worker2 # [ 232.257210] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.856Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7021worker2 # [ 232.258765] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.856Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7022lb # [ 232.253926] niks3-server[1138]: 2026/09/22 11:02:17 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7023worker2 # [ 232.289669] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.888Z level=INFO msg="Uploading 1 narinfos"7024lb # [ 232.268887] niks3-server[1138]: 2026/09/22 11:02:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7025lb # [ 232.281777] niks3-server[1138]: 2026/09/22 11:02:17 INFO Signed narinfos id=78 count=17026lb # [ 232.287743] niks3-server[1138]: 2026/09/22 11:02:17 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7027lb # [ 232.295205] niks3-server[1138]: 2026/09/22 11:02:17 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7028lb # [ 232.301676] niks3-server[1138]: 2026/09/22 11:02:17 INFO Completed upload id=787029worker2 # [ 232.323218] nix-grpc-daemon[2032]: time=2026-09-22T11:02:17.922Z level=INFO msg="Upload complete. (376ms)"7030lb # [ 232.307269] niks3-server[1138]: 2026/09/22 11:02:17 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7031worker2 # [ 232.332560] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33364 duration_s=0 paths=1 nar_bytes_in=7607032worker2 # [ 232.346736] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333647033worker2 # [ 232.353643] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)7034worker2 # [ 232.359498] nix-daemon[855]: reaped child process 2113, status = succeeded7035client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7036worker2 # [ 232.372909] nix-daemon[855]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)7037client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7038client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7039client # farm-top> LOG-farm-top-drain7040worker2 # [ 233.267714] nix-daemon[855]: reaped child process 2127, status = succeeded7041worker2 # [ 233.354413] nix-daemon[855]: accepted connection from pid 2143, user nix-grpc-daemon (trusted)7042worker2 # [ 233.377971] nix-daemon[855]: reaped child process 2146, status = succeeded7043lb # [ 233.361293] niks3-server[1138]: 2026/09/22 11:02:18 INFO Received uploads request method=POST path=/api/pending_closures7044worker2 # [ 233.394610] nix-grpc-daemon[2032]: time=2026-09-22T11:02:18.991Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7045worker2 # [ 233.396598] nix-grpc-daemon[2032]: time=2026-09-22T11:02:18.991Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7046lb # [ 233.400889] niks3-server[1138]: 2026/09/22 11:02:19 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7047worker2 # [ 233.437672] nix-grpc-daemon[2032]: time=2026-09-22T11:02:19.036Z level=INFO msg="Uploading 1 narinfos"7048lb # [ 233.417103] niks3-server[1138]: 2026/09/22 11:02:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7049lb # [ 233.425387] niks3-server[1138]: 2026/09/22 11:02:19 INFO Signed narinfos id=79 count=17050lb # [ 233.433029] niks3-server[1138]: 2026/09/22 11:02:19 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7051lb # [ 233.444181] niks3-server[1138]: 2026/09/22 11:02:19 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7052lb # [ 233.447932] niks3-server[1138]: 2026/09/22 11:02:19 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7053lb # [ 233.456674] niks3-server[1138]: 2026/09/22 11:02:19 INFO Completed upload id=797054worker2 # [ 233.476432] nix-grpc-daemon[2032]: time=2026-09-22T11:02:19.075Z level=INFO msg="Upload complete. (290ms)"7055lb # [ 233.458855] niks3-server[1138]: 2026/09/22 11:02:19 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7056worker2 # [ 233.489257] nix-grpc-daemon[1972]: ts=2026-09-22T11:02:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33364 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=17057worker2 # [ 233.508473] nix-daemon[855]: reaped child process 2125, status = succeeded7058client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 7.88 seconds)7059client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7060client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7061worker1: must succeed: rm /nix/.rw-store/fill7062worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7063worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7064worker1 # [ 234.863566] nix-grpc-daemon[1502]: ts=2026-09-22T11:02:20Z level=info event=healthy reason=min_free7065worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)7066(finished: subtest: low disk drains a worker and builds go to the other, in 11.70 seconds)7067(finished: run the VM test script, in 235.97 seconds)7068test script finished in 236.16s7069cleanup7070kill QemuMachine (pid 49)7071client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7072client # [2026-09-22T11:02:20Z INFO virtiofsd] Client disconnected, shutting down7073client # [2026-09-22T11:02:20Z INFO virtiofsd] Client disconnected, shutting down7074client # [2026-09-22T11:02:20Z INFO virtiofsd] Client disconnected, shutting down7075kill QemuMachine (pid 50)7076lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7077lb # [2026-09-22T11:02:20Z INFO virtiofsd] Client disconnected, shutting down7078lb # [2026-09-22T11:02:20Z INFO virtiofsd] Client disconnected, shutting down7079lb # [2026-09-22T11:02:20Z INFO virtiofsd] Client disconnected, shutting down7080kill QemuMachine (pid 51)7081worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7082worker1 # [2026-09-22T11:02:21Z INFO virtiofsd] Client disconnected, shutting down7083worker1 # [2026-09-22T11:02:21Z INFO virtiofsd] Client disconnected, shutting down7084worker1 # [2026-09-22T11:02:21Z INFO virtiofsd] Client disconnected, shutting down7085kill QemuMachine (pid 52)7086worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7087worker2 # [2026-09-22T11:02:21Z INFO virtiofsd] Client disconnected, shutting down7088worker2 # [2026-09-22T11:02:21Z INFO virtiofsd] Client disconnected, shutting down7089worker2 # [2026-09-22T11:02:21Z INFO virtiofsd] Client disconnected, shutting down7090(finished: cleanup, in 1.19 seconds)7091additionally exposed symbols:7092 client, lb, worker1, worker2,7093 vlan1,7094 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh