vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #497
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client: QEMU running (pid 49)14client # Formatting '/build/vm-state-client/tmp.t8W6lP5d83', fmt=raw size=107374182415worker2: QEMU running (pid 52)16worker1 # Disk image does not exist, creating the virtualisation disk image...17lb: QEMU running (pid 50)18client # mke2fs 1.47.4 (6-Mar-2025)19lb # Disk image does not exist, creating the virtualisation disk image...20worker2 # Disk image does not exist, creating the virtualisation disk image...21client # Discarding device blocks: 0/262144 done22lb # Formatting '/build/vm-state-lb/tmp.5o23et6ELD', fmt=raw size=107374182423worker2 # Formatting '/build/vm-state-worker2/tmp.6jShamaDoo', fmt=raw size=107374182424client # Creating filesystem with 262144 4k blocks and 65536 inodes25lb # mke2fs 1.47.4 (6-Mar-2025)26worker2 # mke2fs 1.47.4 (6-Mar-2025)27client # Filesystem UUID: bd498c12-dff1-4cc1-bc1d-be1cb8394cc728lb # Discarding device blocks: 0/262144 done29worker2 # Discarding device blocks: 0/262144 done30client # Superblock backups stored on blocks:31lb # Creating filesystem with 262144 4k blocks and 65536 inodes32worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes33client # 32768, 98304, 163840, 22937634lb # Filesystem UUID: 209df36f-c74f-438a-b883-feb5929c534135worker2 # Filesystem UUID: 80cb3d53-321a-4129-8263-8fcad6d40ced36client # 37lb # Superblock backups stored on blocks:38worker2 # Superblock backups stored on blocks:39client # Allocating group tables: 0/8 done40lb # 32768, 98304, 163840, 22937641worker2 # 32768, 98304, 163840, 22937642client # Writing inode tables: 0/8 done43lb # 44worker2 # 45client # Creating journal (8192 blocks): done46lb # Allocating group tables: 0/8 done47worker2 # Allocating group tables: 0/8 done48client # Writing superblocks and filesystem accounting information: 0/8 done49lb # Writing inode tables: 0/8 done50worker2 # Writing inode tables: 0/8 done51client # 52lb # Creating journal (8192 blocks): done53worker2 # Creating journal (8192 blocks): done54client # Virtualisation disk image created.55lb # Writing superblocks and filesystem accounting information: 0/8 done56worker2 # Writing superblocks and filesystem accounting information: 0/8 done57client # Starting virtiofs daemons...58lb # 59worker2 # 60client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)61lb # Virtualisation disk image created.62worker2 # Virtualisation disk image created.63client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether64lb # Starting virtiofs daemons...65worker2 # Starting virtiofs daemons...66client # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...67lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)68worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73lb # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...74worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...75client # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...76lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether80worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether81client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82lb # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...83worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...84client # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...85lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87client # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests88lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90client # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests91lb # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...92worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...93client # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests94lb # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests95worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests96worker1: QEMU running (pid 51)97worker1 # Formatting '/build/vm-state-worker1/tmp.PFbKYLG5Vq', fmt=raw size=107374182498lb # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests99worker1 # mke2fs 1.47.4 (6-Mar-2025)100lb # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests101worker1 # Discarding device blocks: 0/262144 done102(finished: start all VMs, in 0.52 seconds)103worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes104worker1 # Filesystem UUID: 9c69db1f-c800-4716-bd16-1a58065274da105client: waiting for unit network-addresses-eth1.service106worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests107client: waiting for the VM to finish booting108worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests109worker1 # Superblock backups stored on blocks:110worker1 # 32768, 98304, 163840, 229376111worker1 # 112worker1 # Allocating group tables: 0/8 done113worker1 # Writing inode tables: 0/8 done114worker1 # Creating journal (8192 blocks): done115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135client # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140client # [ 0.000000] Zone ranges:141client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] DMA32 empty143client # [ 0.000000] Normal empty144client # [ 0.000000] Device empty145client # [ 0.000000] Movable zone start for each node146client # [ 0.000000] Early memory node ranges147client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150client # [ 0.000000] psci: probing for conduit method from DT.151client # [ 0.000000] psci: PSCIv1.3 detected in firmware.152client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153client # [ 0.000000] psci: Trusted OS migration not required154client # [ 0.000000] psci: SMC Calling Convention v1.1155client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157client # [ 0.000000] Detected PIPT I-cache on CPU0158client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159client # [ 0.000000] CPU features: detected: GICv3 CPU interface160client # [ 0.000000] CPU features: detected: Spectre-v4161client # [ 0.000000] CPU features: detected: Spectre-BHB162worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]163client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38164client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23165client # [ 0.000000] alternatives: applying boot alternatives166worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026167worker2 # [ 0.000000] KASLR enabled168worker2 # [ 0.000000] random: crng init done169worker2 # [ 0.000000] Machine model: linux,dummy-virt170worker2 # [ 0.000000] efi: UEFI not found.171worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT172worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]173worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]174client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test/init regInfo=/nix/store/7pzypp1k8pcxgz5kigmsrkgswbnh2dyi-closure-info/registration console=ttyAMA0,115200n8 console=tty0175worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]176worker2 # [ 0.000000] Zone ranges:177worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026178worker1 # [ 0.000000] KASLR enabled179worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]180client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/7pzypp1k8pcxgz5kigmsrkgswbnh2dyi-closure-info/registration", will be passed to user space.181worker2 # [ 0.000000] DMA32 empty182worker1 # [ 0.000000] random: crng init done183worker2 # [ 0.000000] Normal empty184worker1 # [ 0.000000] Machine model: linux,dummy-virt185worker2 # [ 0.000000] Device empty186client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes187worker1 # [ 0.000000] efi: UEFI not found.188worker2 # [ 0.000000] Movable zone start for each node189worker2 # [ 0.000000] Early memory node ranges190client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)191worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT192worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]193worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]194client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)195worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]196worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]197client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB198worker1 # [ 0.000000] Zone ranges:199worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000200client # [ 0.000000] software IO TLB: area num 1.201worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]202worker2 # [ 0.000000] psci: probing for conduit method from DT.203client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB204worker1 # [ 0.000000] DMA32 empty205worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.206worker1 # [ 0.000000] Normal empty207worker1 # [ 0.000000] Device empty208client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)209worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs210worker1 # [ 0.000000] Movable zone start for each node211client # [ 0.000000] Fallback order for Node 0: 0212worker2 # [ 0.000000] psci: Trusted OS migration not required213worker1 # [ 0.000000] Early memory node ranges214worker2 # [ 0.000000] psci: SMC Calling Convention v1.1215client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216216worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]217client # [ 0.000000] Policy zone: DMA218worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)219worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]220client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off221worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000222worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296223client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1224worker2 # [ 0.000000] Detected PIPT I-cache on CPU0225worker1 # [ 0.000000] psci: probing for conduit method from DT.226client # [ 0.000000] allocated 3145728 bytes of page_ext227worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.228client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages229worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)230worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs231client # [ 0.000000] ftrace: allocated 294 pages with 4 groups232worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface233worker1 # [ 0.000000] psci: Trusted OS migration not required234client # [ 0.000000] rcu: Hierarchical RCU implementation.235worker2 # [ 0.000000] CPU features: detected: Spectre-v4236worker1 # [ 0.000000] psci: SMC Calling Convention v1.1237client # [ 0.000000] rcu: RCU event tracing is enabled.238worker2 # [ 0.000000] CPU features: detected: Spectre-BHB239worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)240client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.241worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38242client # [ 0.000000] Trampoline variant of Tasks RCU enabled.243worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296244worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23245client # [ 0.000000] Rude variant of Tasks RCU enabled.246worker1 # [ 0.000000] Detected PIPT I-cache on CPU0247worker2 # [ 0.000000] alternatives: applying boot alternatives248client # [ 0.000000] Tracing variant of Tasks RCU enabled.249worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)250client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.251worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface252worker1 # [ 0.000000] CPU features: detected: Spectre-v4253client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1254worker1 # [ 0.000000] CPU features: detected: Spectre-BHB255worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38256client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.257worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fjl79kv3mslif1ryinjddfhvml6nj1zy-nixos-system-worker2-test/init regInfo=/nix/store/5w3f1zdp329rh9c37p2qh6m4v14r7217-closure-info/registration console=ttyAMA0,115200n8 console=tty0258worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23259client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.260worker1 # [ 0.000000] alternatives: applying boot alternatives261client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.262worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/5w3f1zdp329rh9c37p2qh6m4v14r7217-closure-info/registration", will be passed to user space.263client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0264worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes265client # [ 0.000000] GICv3: 256 SPIs implemented266client # [ 0.000000] GICv3: 0 Extended SPIs implemented267worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)268client # [ 0.000000] Root IRQ handler: gic_handle_irq269worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7w3hl2ifzqxyp2m0j26ag73ir8lww0pf-nixos-system-worker1-test/init regInfo=/nix/store/83grcsshvji6snxndrs5k1lrxmsrsjr5-closure-info/registration console=ttyAMA0,115200n8 console=tty0270worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)271client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI272client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0273worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB274worker2 # [ 0.000000] software IO TLB: area num 1.275client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000276worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/83grcsshvji6snxndrs5k1lrxmsrsjr5-closure-info/registration", will be passed to user space.277client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]278worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB279worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes280worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)281client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)282worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)283worker2 # [ 0.000000] Fallback order for Node 0: 0284client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)285worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216286worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)287worker2 # [ 0.000000] Policy zone: DMA288client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000289worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB290worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off291worker1 # [ 0.000000] software IO TLB: area num 1.292client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000293worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1294worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB295client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.296worker2 # [ 0.000000] allocated 3145728 bytes of page_ext297worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)298worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages299worker1 # [ 0.000000] Fallback order for Node 0: 0300client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns301worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups302client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).303worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.304worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216305worker1 # [ 0.000000] Policy zone: DMA306worker2 # [ 0.000000] rcu: RCU event tracing is enabled.307worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off308client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns309worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.310worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1311worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.312client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns313worker1 # [ 0.000000] allocated 3145728 bytes of page_ext314worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.315client # [ 0.000037] arm-pv: using stolen time PV316worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.317worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages318worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups319client # [ 0.000665] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)320worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.321worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.322client # [ 0.000871] Console: colour dummy device 80x25323worker1 # [ 0.000000] rcu: RCU event tracing is enabled.324worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1325client # [ 0.000879] printk: legacy console [tty0] enabled326worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.327worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.328worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.329client # [ 0.001096] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)330worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.331client # [ 0.001102] pid_max: default: 32768 minimum: 301332worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.333worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.334client # [ 0.001177] LSM: initializing lsm=capability,landlock,yama,bpf,ima335client # [ 0.001348] landlock: Up and running.336worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.337worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.338client # [ 0.001351] Yama: becoming mindful.339worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0340client # [ 0.002008] LSM support for eBPF active341worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1342worker2 # [ 0.000000] GICv3: 256 SPIs implemented343worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented344client # [ 0.002185] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)345worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.346worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq347client # [ 0.002225] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)348worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI349worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.350client # [ 0.004221] rcu: Hierarchical SRCU implementation.351worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0352client # [ 0.004226] rcu: Max phase no-delay instances is 1000.353lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]354worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.355worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000356client # [ 0.005502] fsl-mc MSI: its@8080000 domain created357worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]358worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0359client # [ 0.005593] EFI services will not be available.360worker1 # [ 0.000000] GICv3: 256 SPIs implemented361client # [ 0.005682] smp: Bringing up secondary CPUs ...362lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026363worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)364worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented365lb # [ 0.000000] KASLR enabled366client # [ 0.005690] smp: Brought up 1 node, 1 CPU367worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq368lb # [ 0.000000] random: crng init done369client # [ 0.005694] SMP: Total of 1 processors activated.370lb # [ 0.000000] Machine model: linux,dummy-virt371worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI372worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)373client # [ 0.005699] CPU: All CPU(s) started at EL1374lb # [ 0.000000] efi: UEFI not found.375worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0376worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000377client # [ 0.005709] CPU features: detected: Branch Target Identification378lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT379worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000380client # [ 0.005713] CPU features: detected: ARMv8.4 Translation Table Level381worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000382worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]383lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]384worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.385client # [ 0.005716] CPU features: detected: Instruction cache invalidation not required for I/D coherence386lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]387worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)388lb # [ 0.000000] Zone ranges:389worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns390client # [ 0.005720] CPU features: detected: Data cache clean to the PoU not required for I/D coherence391lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]392worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)393worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).394lb # [ 0.000000] DMA32 empty395client # [ 0.005724] CPU features: detected: Common not Private translations396lb # [ 0.000000] Normal empty397worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000398lb # [ 0.000000] Device empty399client # [ 0.005727] CPU features: detected: CRC32 instructions400lb # [ 0.000000] Movable zone start for each node401worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns402worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000403lb # [ 0.000000] Early memory node ranges404client # [ 0.005730] CPU features: detected: Data cache clean to Point of Deep Persistence405worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns406worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.407lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]408client # [ 0.005734] CPU features: detected: Data cache clean to Point of Persistence409worker2 # [ 0.000032] arm-pv: using stolen time PV410lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]411client # [ 0.005737] CPU features: detected: Data independent timing control (DIT)412worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns413lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000414client # [ 0.005740] CPU features: detected: E0PD415worker2 # [ 0.000450] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)416worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).417lb # [ 0.000000] psci: probing for conduit method from DT.418worker2 # [ 0.000617] Console: colour dummy device 80x25419client # [ 0.005743] CPU features: detected: Enhanced Counter Virtualization420lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.421worker2 # [ 0.000624] printk: legacy console [tty0] enabled422client # [ 0.005746] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)423worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns424lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs425client # [ 0.005750] CPU features: detected: Enhanced Virtualization Traps426lb # [ 0.000000] psci: Trusted OS migration not required427worker2 # [ 0.000838] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)428worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns429lb # [ 0.000000] psci: SMC Calling Convention v1.1430client # [ 0.005753] CPU features: detected: Fine Grained Traps431worker2 # [ 0.000845] pid_max: default: 32768 minimum: 301432worker1 # [ 0.000031] arm-pv: using stolen time PV433worker2 # [ 0.000917] LSM: initializing lsm=capability,landlock,yama,bpf,ima434lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)435client # [ 0.005757] CPU features: detected: Generic authentication (architected QARMA5 algorithm)436worker2 # [ 0.001055] landlock: Up and running.437worker1 # [ 0.000455] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)438client # [ 0.005763] CPU features: detected: RCpc load-acquire (LDAPR)439worker2 # [ 0.001058] Yama: becoming mindful.440lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296441worker1 # [ 0.000653] Console: colour dummy device 80x25442worker2 # [ 0.001550] LSM support for eBPF active443client # [ 0.005766] CPU features: detected: LSE atomic instructions444lb # [ 0.000000] Detected PIPT I-cache on CPU0445worker1 # [ 0.000660] printk: legacy console [tty0] enabled446client # [ 0.005768] CPU features: detected: Privileged Access Never447worker2 # [ 0.001701] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)448lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)449client # [ 0.005771] CPU features: detected: PMUv3450worker1 # [ 0.000851] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)451lb # [ 0.000000] CPU features: detected: GICv3 CPU interface452worker2 # [ 0.001734] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)453client # [ 0.005774] CPU features: detected: RAS Extension Support454worker1 # [ 0.000858] pid_max: default: 32768 minimum: 301455lb # [ 0.000000] CPU features: detected: Spectre-v4456worker2 # [ 0.003586] rcu: Hierarchical SRCU implementation.457client # [ 0.005777] CPU features: detected: RASv1p1 Extension Support458lb # [ 0.000000] CPU features: detected: Spectre-BHB459worker1 # [ 0.000939] LSM: initializing lsm=capability,landlock,yama,bpf,ima460worker2 # [ 0.003591] rcu: Max phase no-delay instances is 1000.461worker1 # [ 0.001077] landlock: Up and running.462client # [ 0.005780] CPU features: detected: Random Number Generator463lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38464worker2 # [ 0.004858] fsl-mc MSI: its@8080000 domain created465worker1 # [ 0.001080] Yama: becoming mindful.466worker1 # [ 0.001564] LSM support for eBPF active467worker2 # [ 0.004950] EFI services will not be available.468lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23469client # [ 0.005782] CPU features: detected: Speculation barrier (SB)470worker2 # [ 0.005060] smp: Bringing up secondary CPUs ...471lb # [ 0.000000] alternatives: applying boot alternatives472worker1 # [ 0.001726] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)473worker2 # [ 0.005068] smp: Brought up 1 node, 1 CPU474client # [ 0.005785] CPU features: detected: Stage-2 Force Write-Back475worker2 # [ 0.005070] SMP: Total of 1 processors activated.476worker1 # [ 0.001759] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)477client # [ 0.005787] CPU features: detected: TLB range maintenance instructions478worker2 # [ 0.005076] CPU: All CPU(s) started at EL1479worker1 # [ 0.003653] rcu: Hierarchical SRCU implementation.480worker2 # [ 0.005087] CPU features: detected: Branch Target Identification481client # [ 0.005792] CPU features: detected: Speculative Store Bypassing Safe (SSBS)482worker1 # [ 0.003658] rcu: Max phase no-delay instances is 1000.483worker2 # [ 0.005092] CPU features: detected: ARMv8.4 Translation Table Level484client # [ 0.005833] alternatives: applying system-wide alternatives485worker1 # [ 0.004959] fsl-mc MSI: its@8080000 domain created486worker1 # [ 0.005049] EFI services will not be available.487lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test/init regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration console=ttyAMA0,115200n8 console=tty0488client # [ 0.008843] CPU features: detected: BBM Level 2 without TLB conflict abort489worker2 # [ 0.005095] CPU features: detected: Instruction cache invalidation not required for I/D coherence490worker1 # [ 0.005130] smp: Bringing up secondary CPUs ...491worker1 # [ 0.005139] smp: Brought up 1 node, 1 CPU492worker2 # [ 0.005099] CPU features: detected: Data cache clean to the PoU not required for I/D coherence493worker1 # [ 0.005141] SMP: Total of 1 processors activated.494lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration", will be passed to user space.495client # [ 0.009003] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)496worker1 # [ 0.005147] CPU: All CPU(s) started at EL1497client # [ 0.009429] devtmpfs: initialized498worker2 # [ 0.005103] CPU features: detected: Common not Private translations499lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes500worker1 # [ 0.005157] CPU features: detected: Branch Target Identification501worker2 # [ 0.005106] CPU features: detected: CRC32 instructions502client # [ 0.011293] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)503lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)504worker1 # [ 0.005162] CPU features: detected: ARMv8.4 Translation Table Level505worker2 # [ 0.005109] CPU features: detected: Data cache clean to Point of Deep Persistence506client # [ 0.011316] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).507lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)508worker2 # [ 0.005113] CPU features: detected: Data cache clean to Point of Persistence509worker1 # [ 0.005165] CPU features: detected: Instruction cache invalidation not required for I/D coherence510client # [ 0.011512] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL511lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB512worker2 # [ 0.005116] CPU features: detected: Data independent timing control (DIT)513client # [ 0.011516] 0 pages in range for non-PLT usage514lb # [ 0.000000] software IO TLB: area num 1.515worker1 # [ 0.005168] CPU features: detected: Data cache clean to the PoU not required for I/D coherence516worker2 # [ 0.005119] CPU features: detected: E0PD517client # [ 0.011517] 508272 pages in range for PLT usage518worker1 # [ 0.005172] CPU features: detected: Common not Private translations519lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB520client # [ 0.011638] pinctrl core: initialized pinctrl subsystem521worker2 # [ 0.005121] CPU features: detected: Enhanced Counter Virtualization522worker1 # [ 0.005176] CPU features: detected: CRC32 instructions523client # [ 0.012372] DMI not present or invalid.524lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)525worker2 # [ 0.005125] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)526client # [ 0.015951] NET: Registered PF_NETLINK/PF_ROUTE protocol family527lb # [ 0.000000] Fallback order for Node 0: 0528worker1 # [ 0.005178] CPU features: detected: Data cache clean to Point of Deep Persistence529worker2 # [ 0.005128] CPU features: detected: Enhanced Virtualization Traps530lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216531client # [ 0.018342] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations532worker1 # [ 0.005182] CPU features: detected: Data cache clean to Point of Persistence533worker2 # [ 0.005131] CPU features: detected: Fine Grained Traps534lb # [ 0.000000] Policy zone: DMA535client # [ 0.018535] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations536worker1 # [ 0.005185] CPU features: detected: Data independent timing control (DIT)537lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off538worker2 # [ 0.005135] CPU features: detected: Generic authentication (architected QARMA5 algorithm)539worker1 # [ 0.005188] CPU features: detected: E0PD540client # [ 0.018738] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations541worker2 # [ 0.005140] CPU features: detected: RCpc load-acquire (LDAPR)542lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1543worker1 # [ 0.005190] CPU features: detected: Enhanced Counter Virtualization544lb # [ 0.000000] allocated 3145728 bytes of page_ext545client # [ 0.018762] audit: initializing netlink subsys (disabled)546worker2 # [ 0.005143] CPU features: detected: LSE atomic instructions547worker1 # [ 0.005193] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)548lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages549worker2 # [ 0.005146] CPU features: detected: Privileged Access Never550client # [ 0.019323] thermal_sys: Registered thermal governor 'fair_share'551worker2 # [ 0.005148] CPU features: detected: PMUv3552lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups553worker1 # [ 0.005197] CPU features: detected: Enhanced Virtualization Traps554client # [ 0.019325] thermal_sys: Registered thermal governor 'bang_bang'555lb # [ 0.000000] rcu: Hierarchical RCU implementation.556worker1 # [ 0.005200] CPU features: detected: Fine Grained Traps557worker2 # [ 0.005151] CPU features: detected: RAS Extension Support558client # [ 0.019329] thermal_sys: Registered thermal governor 'step_wise'559lb # [ 0.000000] rcu: RCU event tracing is enabled.560worker2 # [ 0.005154] CPU features: detected: RASv1p1 Extension Support561worker1 # [ 0.005204] CPU features: detected: Generic authentication (architected QARMA5 algorithm)562client # [ 0.019332] thermal_sys: Registered thermal governor 'user_space'563lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.564worker2 # [ 0.005157] CPU features: detected: Random Number Generator565worker1 # [ 0.005208] CPU features: detected: RCpc load-acquire (LDAPR)566client # [ 0.019337] thermal_sys: Registered thermal governor 'power_allocator'567lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.568worker2 # [ 0.005159] CPU features: detected: Speculation barrier (SB)569worker1 # [ 0.005211] CPU features: detected: LSE atomic instructions570lb # [ 0.000000] Rude variant of Tasks RCU enabled.571worker2 # [ 0.005162] CPU features: detected: Stage-2 Force Write-Back572client # [ 0.019364] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1573worker1 # [ 0.005214] CPU features: detected: Privileged Access Never574lb # [ 0.000000] Tracing variant of Tasks RCU enabled.575client # [ 0.019373] cpuidle: using governor ladder576worker2 # [ 0.005165] CPU features: detected: TLB range maintenance instructions577worker1 # [ 0.005217] CPU features: detected: PMUv3578client # [ 0.019379] cpuidle: using governor menu579lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.580worker1 # [ 0.005220] CPU features: detected: RAS Extension Support581worker2 # [ 0.005170] CPU features: detected: Speculative Store Bypassing Safe (SSBS)582client # [ 0.019562] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.583lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1584worker1 # [ 0.005222] CPU features: detected: RASv1p1 Extension Support585worker2 # [ 0.005209] alternatives: applying system-wide alternatives586client # [ 0.019579] ASID allocator initialised with 65536 entries587worker1 # [ 0.005225] CPU features: detected: Random Number Generator588client # [ 0.020789] Serial: AMBA PL011 UART driver589lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.590worker2 # [ 0.008235] CPU features: detected: BBM Level 2 without TLB conflict abort591client # [ 0.026123] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1592lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.593worker1 # [ 0.005228] CPU features: detected: Speculation barrier (SB)594client # [ 0.026262] printk: console [ttyAMA0] enabled595worker2 # [ 0.008446] Memory: 1405916K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)596worker1 # [ 0.005231] CPU features: detected: Stage-2 Force Write-Back597worker2 # [ 0.008825] devtmpfs: initialized598lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.599worker1 # [ 0.005233] CPU features: detected: TLB range maintenance instructions600lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0601worker2 # [ 0.010938] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)602lb # [ 0.000000] GICv3: 256 SPIs implemented603worker1 # [ 0.005238] CPU features: detected: Speculative Store Bypassing Safe (SSBS)604lb # [ 0.000000] GICv3: 0 Extended SPIs implemented605worker2 # [ 0.010962] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).606worker1 # [ 0.005276] alternatives: applying system-wide alternatives607lb # [ 0.000000] Root IRQ handler: gic_handle_irq608worker2 # [ 0.011155] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL609lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI610worker1 # [ 0.008198] CPU features: detected: BBM Level 2 without TLB conflict abort611lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0612worker2 # [ 0.011160] 0 pages in range for non-PLT usage613lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000614worker1 # [ 0.008366] Memory: 1405624K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)615worker2 # [ 0.011161] 508272 pages in range for PLT usage616lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]617worker1 # [ 0.008757] devtmpfs: initialized618worker2 # [ 0.011270] pinctrl core: initialized pinctrl subsystem619worker2 # [ 0.012145] DMI not present or invalid.620worker1 # [ 0.010603] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)621lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)622worker2 # [ 0.015450] NET: Registered PF_NETLINK/PF_ROUTE protocol family623client # [ 0.145307] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages624worker1 # [ 0.010626] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).625lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)626client # [ 0.145328] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page627worker2 # [ 0.018201] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations628worker1 # [ 0.010793] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL629lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000630worker1 # [ 0.010797] 0 pages in range for non-PLT usage631client # [ 0.145333] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages632worker2 # [ 0.018390] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations633lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000634worker1 # [ 0.010799] 508272 pages in range for PLT usage635client # [ 0.145338] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page636worker2 # [ 0.018596] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations637worker1 # [ 0.010913] pinctrl core: initialized pinctrl subsystem638lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.639client # [ 0.145342] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages640worker2 # [ 0.018620] audit: initializing netlink subsys (disabled)641worker1 # [ 0.011685] DMI not present or invalid.642client # [ 0.145347] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page643worker2 # [ 0.019211] thermal_sys: Registered thermal governor 'fair_share'644worker1 # [ 0.015358] NET: Registered PF_NETLINK/PF_ROUTE protocol family645lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns646worker2 # [ 0.019213] thermal_sys: Registered thermal governor 'bang_bang'647client # [ 0.145351] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages648lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).649worker1 # [ 0.017724] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations650worker2 # [ 0.019217] thermal_sys: Registered thermal governor 'step_wise'651client # [ 0.145355] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page652worker1 # [ 0.017917] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations653lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns654worker2 # [ 0.019220] thermal_sys: Registered thermal governor 'user_space'655worker1 # [ 0.018124] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations656worker2 # [ 0.019225] thermal_sys: Registered thermal governor 'power_allocator'657lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns658worker1 # [ 0.018148] audit: initializing netlink subsys (disabled)659lb # [ 0.000031] arm-pv: using stolen time PV660client # [ 0.152637] fbcon: Taking over console661worker2 # [ 0.019251] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1662worker1 # [ 0.018747] thermal_sys: Registered thermal governor 'fair_share'663client # [ 0.152652] ACPI: Interpreter disabled.664worker2 # [ 0.019259] cpuidle: using governor ladder665lb # [ 0.000390] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)666worker1 # [ 0.018749] thermal_sys: Registered thermal governor 'bang_bang'667worker2 # [ 0.019265] cpuidle: using governor menu668lb # [ 0.000603] Console: colour dummy device 80x25669worker1 # [ 0.018753] thermal_sys: Registered thermal governor 'step_wise'670worker2 # [ 0.019461] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.671lb # [ 0.000611] printk: legacy console [tty0] enabled672worker1 # [ 0.018755] thermal_sys: Registered thermal governor 'user_space'673worker2 # [ 0.019478] ASID allocator initialised with 65536 entries674worker2 # [ 0.020732] Serial: AMBA PL011 UART driver675lb # [ 0.000799] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)676worker1 # [ 0.018761] thermal_sys: Registered thermal governor 'power_allocator'677lb # [ 0.000806] pid_max: default: 32768 minimum: 301678worker2 # [ 0.025984] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1679worker1 # [ 0.018785] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1680lb # [ 0.000888] LSM: initializing lsm=capability,landlock,yama,bpf,ima681worker2 # [ 0.026122] printk: console [ttyAMA0] enabled682worker1 # [ 0.018793] cpuidle: using governor ladder683lb # [ 0.001009] landlock: Up and running.684worker1 # [ 0.018799] cpuidle: using governor menu685lb # [ 0.001012] Yama: becoming mindful.686lb # [ 0.001446] LSM support for eBPF active687worker1 # [ 0.019001] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.688worker1 # [ 0.019017] ASID allocator initialised with 65536 entries689lb # [ 0.001588] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)690worker1 # [ 0.020227] Serial: AMBA PL011 UART driver691lb # [ 0.001622] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)692worker1 # [ 0.025531] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1693lb # [ 0.003528] rcu: Hierarchical SRCU implementation.694worker1 # [ 0.025701] printk: console [ttyAMA0] enabled695lb # [ 0.003534] rcu: Max phase no-delay instances is 1000.696client # [ 0.161677] iommu: Default domain type: Translated697lb # [ 0.004799] fsl-mc MSI: its@8080000 domain created698client # [ 0.161688] iommu: DMA domain TLB invalidation policy: strict mode699lb # [ 0.004896] EFI services will not be available.700client # [ 0.162031] SCSI subsystem initialized701lb # [ 0.004979] smp: Bringing up secondary CPUs ...702lb # [ 0.004987] smp: Brought up 1 node, 1 CPU703worker2 # [ 0.144246] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages704client # [ 0.164023] usbcore: registered new interface driver usbfs705lb # [ 0.004991] SMP: Total of 1 processors activated.706lb # [ 0.004996] CPU: All CPU(s) started at EL1707worker2 # [ 0.144266] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page708client # [ 0.164069] usbcore: registered new interface driver hub709client # [ 0.164086] usbcore: registered new device driver usb710lb # [ 0.005006] CPU features: detected: Branch Target Identification711worker2 # [ 0.144271] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages712client # [ 0.164361] pps_core: LinuxPPS API ver. 1 registered713lb # [ 0.005010] CPU features: detected: ARMv8.4 Translation Table Level714worker2 # [ 0.144275] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page715client # [ 0.164367] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>716worker2 # [ 0.144280] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages717lb # [ 0.005013] CPU features: detected: Instruction cache invalidation not required for I/D coherence718client # [ 0.164385] PTP clock support registered719worker2 # [ 0.144284] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page720client # [ 0.164435] EDAC MC: Ver: 3.0.0721worker1 # [ 0.143003] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages722lb # [ 0.005017] CPU features: detected: Data cache clean to the PoU not required for I/D coherence723worker2 # [ 0.144289] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages724client # [ 0.169000] scmi_core: SCMI protocol bus registered725worker1 # [ 0.143020] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page726lb # [ 0.005020] CPU features: detected: Common not Private translations727worker2 # [ 0.144293] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page728lb # [ 0.005024] CPU features: detected: CRC32 instructions729worker1 # [ 0.143025] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages730client # [ 0.169955] FPGA manager framework731client # [ 0.170912] vgaarb: loaded732worker1 # [ 0.143030] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page733lb # [ 0.005027] CPU features: detected: Data cache clean to Point of Deep Persistence734worker1 # [ 0.143034] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages735lb # [ 0.005030] CPU features: detected: Data cache clean to Point of Persistence736client # [ 0.171527] clocksource: Switched to clocksource arch_sys_counter737worker2 # [ 0.151661] fbcon: Taking over console738worker1 # [ 0.143039] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page739lb # [ 0.005033] CPU features: detected: Data independent timing control (DIT)740worker2 # [ 0.151677] ACPI: Interpreter disabled.741lb # [ 0.005036] CPU features: detected: E0PD742worker1 # [ 0.143043] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages743lb # [ 0.005039] CPU features: detected: Enhanced Counter Virtualization744worker1 # [ 0.143047] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page745lb # [ 0.005042] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)746lb # [ 0.005045] CPU features: detected: Enhanced Virtualization Traps747lb # [ 0.005049] CPU features: detected: Fine Grained Traps748worker1 # [ 0.150395] fbcon: Taking over console749worker1 # [ 0.150410] ACPI: Interpreter disabled.750lb # [ 0.005052] CPU features: detected: Generic authentication (architected QARMA5 algorithm)751lb # [ 0.005057] CPU features: detected: RCpc load-acquire (LDAPR)752client # [ 0.177872] VFS: Disk quotas dquot_6.6.0753lb # [ 0.005061] CPU features: detected: LSE atomic instructions754client # [ 0.177917] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)755lb # [ 0.005063] CPU features: detected: Privileged Access Never756lb # [ 0.005066] CPU features: detected: PMUv3757client # [ 0.179646] netfs: FS-Cache loaded758lb # [ 0.005069] CPU features: detected: RAS Extension Support759client # [ 0.179771] pnp: PnP ACPI: disabled760lb # [ 0.005071] CPU features: detected: RASv1p1 Extension Support761worker2 # [ 0.160637] iommu: Default domain type: Translated762lb # [ 0.005074] CPU features: detected: Random Number Generator763worker2 # [ 0.160647] iommu: DMA domain TLB invalidation policy: strict mode764worker2 # [ 0.160975] SCSI subsystem initialized765lb # [ 0.005077] CPU features: detected: Speculation barrier (SB)766lb # [ 0.005079] CPU features: detected: Stage-2 Force Write-Back767worker2 # [ 0.163003] usbcore: registered new interface driver usbfs768lb # [ 0.005083] CPU features: detected: TLB range maintenance instructions769worker2 # [ 0.163035] usbcore: registered new interface driver hub770lb # [ 0.005087] CPU features: detected: Speculative Store Bypassing Safe (SSBS)771worker2 # [ 0.163059] usbcore: registered new device driver usb772worker1 # [ 0.159401] iommu: Default domain type: Translated773lb # [ 0.005124] alternatives: applying system-wide alternatives774worker2 # [ 0.163309] pps_core: LinuxPPS API ver. 1 registered775worker1 # [ 0.159411] iommu: DMA domain TLB invalidation policy: strict mode776lb # [ 0.008119] CPU features: detected: BBM Level 2 without TLB conflict abort777worker1 # [ 0.159745] SCSI subsystem initialized778worker2 # [ 0.163315] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>779worker2 # [ 0.163324] PTP clock support registered780worker2 # [ 0.163372] EDAC MC: Ver: 3.0.0781worker1 # [ 0.161724] usbcore: registered new interface driver usbfs782lb # [ 0.008330] Memory: 1405640K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)783client # [ 0.183606] NET: Registered PF_INET protocol family784lb # [ 0.008756] devtmpfs: initialized785worker1 # [ 0.161757] usbcore: registered new interface driver hub786worker2 # [ 0.168026] scmi_core: SCMI protocol bus registered787client # [ 0.183885] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)788worker1 # [ 0.161772] usbcore: registered new device driver usb789lb # [ 0.010657] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)790worker2 # [ 0.168980] FPGA manager framework791worker1 # [ 0.162042] pps_core: LinuxPPS API ver. 1 registered792worker2 # [ 0.169932] vgaarb: loaded793lb # [ 0.010680] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).794worker1 # [ 0.162048] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>795lb # [ 0.010857] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL796worker2 # [ 0.170557] clocksource: Switched to clocksource arch_sys_counter797worker1 # [ 0.162057] PTP clock support registered798lb # [ 0.010861] 0 pages in range for non-PLT usage799worker1 # [ 0.162104] EDAC MC: Ver: 3.0.0800lb # [ 0.010863] 508272 pages in range for PLT usage801worker1 # [ 0.166643] scmi_core: SCMI protocol bus registered802lb # [ 0.010974] pinctrl core: initialized pinctrl subsystem803lb # [ 0.011776] DMI not present or invalid.804worker1 # [ 0.167633] FPGA manager framework805lb # [ 0.014908] NET: Registered PF_NETLINK/PF_ROUTE protocol family806worker1 # [ 0.168625] vgaarb: loaded807lb # [ 0.017491] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations808worker1 # [ 0.169256] clocksource: Switched to clocksource arch_sys_counter809lb # [ 0.017683] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations810lb # [ 0.017884] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations811lb # [ 0.017905] audit: initializing netlink subsys (disabled)812worker2 # [ 0.176928] VFS: Disk quotas dquot_6.6.0813worker2 # [ 0.176975] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)814lb # [ 0.018469] thermal_sys: Registered thermal governor 'fair_share'815worker2 # [ 0.178685] netfs: FS-Cache loaded816lb # [ 0.018471] thermal_sys: Registered thermal governor 'bang_bang'817worker2 # [ 0.178806] pnp: PnP ACPI: disabled818worker1 # [ 0.173872] VFS: Disk quotas dquot_6.6.0819lb # [ 0.018475] thermal_sys: Registered thermal governor 'step_wise'820worker1 # [ 0.173912] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)821lb # [ 0.018477] thermal_sys: Registered thermal governor 'user_space'822lb # [ 0.018483] thermal_sys: Registered thermal governor 'power_allocator'823lb # [ 0.018508] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1824lb # [ 0.018516] cpuidle: using governor ladder825worker1 # [ 0.177635] netfs: FS-Cache loaded826lb # [ 0.018523] cpuidle: using governor menu827worker1 # [ 0.177757] pnp: PnP ACPI: disabled828lb # [ 0.018714] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.829lb # [ 0.018731] ASID allocator initialised with 65536 entries830lb # [ 0.019947] Serial: AMBA PL011 UART driver831lb # [ 0.025251] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1832worker2 # [ 0.182722] NET: Registered PF_INET protocol family833lb # [ 0.025414] printk: console [ttyAMA0] enabled834worker2 # [ 0.183004] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)835worker1 # [ 0.181640] NET: Registered PF_INET protocol family836worker1 # [ 0.181911] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)837lb # [ 0.144792] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages838lb # [ 0.144811] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page839lb # [ 0.144816] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages840lb # [ 0.144821] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page841lb # [ 0.144825] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages842lb # [ 0.144830] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page843client # [ 0.213027] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)844lb # [ 0.144834] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages845client # [ 0.213073] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)846lb # [ 0.144838] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page847client # [ 0.213099] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)848client # [ 0.213172] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849lb # [ 0.152418] fbcon: Taking over console850client # [ 0.213328] TCP: Hash tables configured (established 16384 bind 16384)851lb # [ 0.152432] ACPI: Interpreter disabled.852client # [ 0.213411] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)853lb # [ 0.154383] iommu: Default domain type: Translated854client # [ 0.213446] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)855lb # [ 0.154393] iommu: DMA domain TLB invalidation policy: strict mode856client # [ 0.213482] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)857lb # [ 0.156092] SCSI subsystem initialized858client # [ 0.213573] NET: Registered PF_UNIX/PF_LOCAL protocol family859client # [ 0.213594] NET: Registered PF_XDP protocol family860client # [ 0.213611] PCI: CLS 0 bytes, default 64861client # [ 0.213857] Trying to unpack rootfs image as initramfs...862client # [ 0.229335] kvm [1]: HYP mode not available863lb # [ 0.161257] usbcore: registered new interface driver usbfs864lb # [ 0.161290] usbcore: registered new interface driver hub865lb # [ 0.161306] usbcore: registered new device driver usb866lb # [ 0.161579] pps_core: LinuxPPS API ver. 1 registered867lb # [ 0.161586] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>868lb # [ 0.161596] PTP clock support registered869lb # [ 0.161663] EDAC MC: Ver: 3.0.0870lb # [ 0.166325] scmi_core: SCMI protocol bus registered871lb # [ 0.167303] FPGA manager framework872lb # [ 0.168270] vgaarb: loaded873lb # [ 0.168911] clocksource: Switched to clocksource arch_sys_counter874worker1 # [ 0.211263] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)875worker2 # [ 0.213095] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)876worker1 # [ 0.211310] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)877lb # [ 0.173780] VFS: Disk quotas dquot_6.6.0878worker2 # [ 0.213140] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)879worker1 # [ 0.211336] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)880lb # [ 0.173817] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)881worker2 # [ 0.213166] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)882worker1 # [ 0.211407] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)883worker2 # [ 0.213237] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884worker1 # [ 0.211550] TCP: Hash tables configured (established 16384 bind 16384)885worker2 # [ 0.213380] TCP: Hash tables configured (established 16384 bind 16384)886worker1 # [ 0.211640] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)887worker2 # [ 0.213462] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)888lb # [ 0.177568] netfs: FS-Cache loaded889worker1 # [ 0.211675] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)890lb # [ 0.177697] pnp: PnP ACPI: disabled891worker2 # [ 0.213515] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)892worker1 # [ 0.211710] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)893worker2 # [ 0.213551] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)894worker1 # [ 0.211800] NET: Registered PF_UNIX/PF_LOCAL protocol family895worker2 # [ 0.213668] NET: Registered PF_UNIX/PF_LOCAL protocol family896worker1 # [ 0.211820] NET: Registered PF_XDP protocol family897worker2 # [ 0.213688] NET: Registered PF_XDP protocol family898worker1 # [ 0.211837] PCI: CLS 0 bytes, default 64899worker2 # [ 0.213706] PCI: CLS 0 bytes, default 64900worker1 # [ 0.212083] Trying to unpack rootfs image as initramfs...901worker2 # [ 0.213945] Trying to unpack rootfs image as initramfs...902worker1 # [ 0.227007] kvm [1]: HYP mode not available903lb # [ 0.181592] NET: Registered PF_INET protocol family904lb # [ 0.181879] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)905worker2 # [ 0.232498] kvm [1]: HYP mode not available906lb # [ 0.212022] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907lb # [ 0.212077] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908lb # [ 0.212104] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909lb # [ 0.212179] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910lb # [ 0.212323] TCP: Hash tables configured (established 16384 bind 16384)911lb # [ 0.212411] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912lb # [ 0.212448] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913lb # [ 0.212484] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914lb # [ 0.212574] NET: Registered PF_UNIX/PF_LOCAL protocol family915lb # [ 0.212613] NET: Registered PF_XDP protocol family916lb # [ 0.212632] PCI: CLS 0 bytes, default 64917lb # [ 0.212885] Trying to unpack rootfs image as initramfs...918lb # [ 0.230662] kvm [1]: HYP mode not available919client # [ 0.325245] Initialise system trusted keyrings920client # [ 0.331576] workingset: timestamp_bits=42 max_order=19 bucket_order=0921client # [ 0.332946] squashfs: version 4.0 (2009/01/31) Phillip Lougher922client # [ 0.333735] 9p: Installing v9fs 9p2000 file system support923worker1 # [ 0.322404] Initialise system trusted keyrings924worker1 # [ 0.323131] workingset: timestamp_bits=42 max_order=19 bucket_order=0925worker2 # [ 0.329017] Initialise system trusted keyrings926worker1 # [ 0.324438] squashfs: version 4.0 (2009/01/31) Phillip Lougher927worker2 # [ 0.329763] workingset: timestamp_bits=42 max_order=19 bucket_order=0928worker1 # [ 0.325208] 9p: Installing v9fs 9p2000 file system support929worker2 # [ 0.331012] squashfs: version 4.0 (2009/01/31) Phillip Lougher930worker2 # [ 0.331761] 9p: Installing v9fs 9p2000 file system support931client # [ 0.354419] Key type asymmetric registered932client # [ 0.354442] Asymmetric key parser 'x509' registered933client # [ 0.354518] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)934client # [ 0.363607] io scheduler mq-deadline registered935client # [ 0.363630] io scheduler kyber registered936client # [ 0.368727] pl061_gpio 9030000.pl061: PL061 GPIO chip registered937client # [ 0.370004] ledtrig-cpu: registered to indicate activity on CPUs938client # [ 0.370385] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:939client # [ 0.370403] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000940client # [ 0.370416] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000941worker2 # [ 0.355779] Key type asymmetric registered942client # [ 0.370424] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000943worker2 # [ 0.355806] Asymmetric key parser 'x509' registered944client # [ 0.370448] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits945worker2 # [ 0.355885] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)946worker2 # [ 0.358051] io scheduler mq-deadline registered947client # [ 0.370477] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]948worker2 # [ 0.358063] io scheduler kyber registered949client # [ 0.370549] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00950worker1 # [ 0.354002] Key type asymmetric registered951client # [ 0.370559] pci_bus 0000:00: root bus resource [bus 00-ff]952worker1 # [ 0.354033] Asymmetric key parser 'x509' registered953client # [ 0.370565] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]954worker1 # [ 0.354140] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)955client # [ 0.370570] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]956worker1 # [ 0.356312] io scheduler mq-deadline registered957client # [ 0.370575] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]958worker1 # [ 0.356323] io scheduler kyber registered959client # [ 0.370628] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint960client # [ 0.371058] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint961client # [ 0.371240] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]962client # [ 0.371256] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]963client # [ 0.371286] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]964client # [ 0.371302] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]965worker2 # [ 0.371271] pl061_gpio 9030000.pl061: PL061 GPIO chip registered966lb # [ 0.325443] Initialise system trusted keyrings967client # [ 0.391853] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint968client # [ 0.392057] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]969lb # [ 0.326201] workingset: timestamp_bits=42 max_order=19 bucket_order=0970client # [ 0.392074] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]971lb # [ 0.327508] squashfs: version 4.0 (2009/01/31) Phillip Lougher972client # [ 0.392104] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973worker2 # [ 0.372617] ledtrig-cpu: registered to indicate activity on CPUs974client # [ 0.392578] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint975lb # [ 0.328301] 9p: Installing v9fs 9p2000 file system support976worker2 # [ 0.373002] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:977client # [ 0.392757] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]978worker2 # [ 0.373019] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000979client # [ 0.392774] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]980client # [ 0.392803] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]981worker2 # [ 0.373031] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000982client # [ 0.393254] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint983worker1 # [ 0.369528] pl061_gpio 9030000.pl061: PL061 GPIO chip registered984worker2 # [ 0.373040] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000985client # [ 0.393448] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]986worker1 # [ 0.370198] ledtrig-cpu: registered to indicate activity on CPUs987worker2 # [ 0.373066] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits988client # [ 0.393464] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]989worker1 # [ 0.370565] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:990client # [ 0.393494] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]991worker2 # [ 0.373102] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]992worker1 # [ 0.370583] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000993client # [ 0.393513] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]994worker2 # [ 0.373187] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00995worker1 # [ 0.370596] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000996worker2 # [ 0.373196] pci_bus 0000:00: root bus resource [bus 00-ff]997client # [ 0.393965] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint998client # [ 0.394152] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]999worker2 # [ 0.373202] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1000worker1 # [ 0.370605] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001001worker2 # [ 0.373207] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1002client # [ 0.394182] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1003worker1 # [ 0.370626] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1004worker2 # [ 0.373212] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1005client # [ 0.394637] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1006worker1 # [ 0.370650] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1007client # [ 0.394824] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1008worker2 # [ 0.373266] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1009worker1 # [ 0.370723] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001010client # [ 0.394854] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1011worker2 # [ 0.373720] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1012worker1 # [ 0.370732] pci_bus 0000:00: root bus resource [bus 00-ff]1013worker2 # [ 0.373905] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1014client # [ 0.395273] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1015worker1 # [ 0.370738] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1016client # [ 0.395453] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1017worker2 # [ 0.373922] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1018worker1 # [ 0.370743] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1019worker2 # [ 0.373952] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1020worker1 # [ 0.370748] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1021worker2 # [ 0.373968] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1022worker1 # [ 0.370829] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1023worker2 # [ 0.374428] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1024worker1 # [ 0.371281] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1025worker2 # [ 0.374636] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1026worker1 # [ 0.371471] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1027worker2 # [ 0.374652] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1028client # [ 0.412634] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1029worker1 # [ 0.371488] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1030client # [ 0.412825] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1031worker2 # [ 0.374682] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1032worker1 # [ 0.371518] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033client # [ 0.412855] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1034worker2 # [ 0.375136] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1035worker1 # [ 0.371535] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1036worker2 # [ 0.375318] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1037client # [ 0.413328] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1038worker1 # [ 0.371998] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1039worker2 # [ 0.375334] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1040client # [ 0.413524] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1041worker1 # [ 0.372205] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1042worker2 # [ 0.375364] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1043client # [ 0.413554] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044worker1 # [ 0.372222] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1045worker2 # [ 0.375817] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1046client # [ 0.414009] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1047worker1 # [ 0.372253] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1048worker2 # [ 0.376000] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1049client # [ 0.414197] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1050worker2 # [ 0.376015] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1051worker1 # [ 0.372728] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1052client # [ 0.414227] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1053worker1 # [ 0.372917] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1054worker2 # [ 0.376045] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1055client # [ 0.414686] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1056worker1 # [ 0.372934] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1057worker2 # [ 0.376063] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1058client # [ 0.414980] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1059worker1 # [ 0.372964] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1060worker2 # [ 0.376521] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1061client # [ 0.414998] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1062worker2 # [ 0.376707] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1063worker1 # [ 0.373455] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1064client # [ 0.415028] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065worker1 # [ 0.373648] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1066worker2 # [ 0.376737] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067client # [ 0.415494] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1068worker1 # [ 0.373664] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker2 # [ 0.377219] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1070worker1 # [ 0.373694] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1071lb # [ 0.357007] Key type asymmetric registered1072worker2 # [ 0.377406] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1073lb # [ 0.357035] Asymmetric key parser 'x509' registered1074worker1 # [ 0.373713] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1075worker2 # [ 0.377436] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1076lb # [ 0.357102] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1077worker1 # [ 0.374175] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1078lb # [ 0.359176] io scheduler mq-deadline registered1079worker2 # [ 0.377842] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1080worker1 # [ 0.374376] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1081lb # [ 0.359187] io scheduler kyber registered1082worker2 # [ 0.378024] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1083worker1 # [ 0.374407] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1084worker2 # [ 0.378285] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1085worker1 # [ 0.374868] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1086worker2 # [ 0.378482] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1087worker1 # [ 0.375060] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1088worker2 # [ 0.378512] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089worker1 # [ 0.375091] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1090worker1 # [ 0.375525] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1091worker1 # [ 0.375712] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1092client # [ 0.431739] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1093worker1 # [ 0.375975] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1094client # [ 0.431767] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1095worker1 # [ 0.376167] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1096client # [ 0.431798] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1097worker1 # [ 0.376213] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098client # [ 0.432416] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1099worker1 # [ 0.376681] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1100client # [ 0.432428] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1101worker1 # [ 0.376874] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1102client # [ 0.432434] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1103worker1 # [ 0.376904] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104lb # [ 0.369075] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1105client # [ 0.432481] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1106client # [ 0.432528] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1107worker2 # [ 0.419091] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1108client # [ 0.432577] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1109lb # [ 0.370494] ledtrig-cpu: registered to indicate activity on CPUs1110worker2 # [ 0.419304] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1111client # [ 0.432625] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1112lb # [ 0.370915] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1113worker2 # [ 0.419336] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114client # [ 0.432673] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1115lb # [ 0.370932] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001116worker2 # [ 0.419792] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1117client # [ 0.432721] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1118worker2 # [ 0.419981] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1119lb # [ 0.370945] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001120worker2 # [ 0.420011] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1121client # [ 0.432768] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1122lb # [ 0.370954] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001123worker2 # [ 0.420473] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1124client # [ 0.432815] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1125lb # [ 0.370973] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1126worker2 # [ 0.420753] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1127client # [ 0.432862] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1128worker2 # [ 0.420770] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1129lb # [ 0.370999] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1130client # [ 0.432980] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1131worker2 # [ 0.420800] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1132lb # [ 0.371093] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001133client # [ 0.433028] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1134worker1 # [ 0.421448] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1135worker2 # [ 0.421261] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1136lb # [ 0.371103] pci_bus 0000:00: root bus resource [bus 00-ff]1137client # [ 0.433050] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1138worker2 # [ 0.421444] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1139worker1 # [ 0.421666] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1140lb # [ 0.371109] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1141worker2 # [ 0.421460] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1142client # [ 0.433071] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1143worker1 # [ 0.421698] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144lb # [ 0.371114] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1145worker2 # [ 0.421507] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146client # [ 0.433093] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1147lb # [ 0.371120] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1148worker1 # [ 0.422196] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1149worker2 # [ 0.422084] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1150client # [ 0.433115] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1151worker1 # [ 0.422488] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1152lb # [ 0.371175] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1153worker1 # [ 0.422505] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1154client # [ 0.433136] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1155worker2 # [ 0.422096] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1156lb # [ 0.371611] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1157client # [ 0.433158] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1158worker2 # [ 0.422101] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1159worker1 # [ 0.422536] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1160lb # [ 0.371797] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1161client # [ 0.433185] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1162worker1 # [ 0.423006] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1163lb # [ 0.371815] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1164worker2 # [ 0.422146] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1165worker1 # [ 0.423197] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1166client # [ 0.433207] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1167lb # [ 0.371845] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1168worker2 # [ 0.422193] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1169worker1 # [ 0.423213] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1170client # [ 0.433230] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1171lb # [ 0.371863] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1172worker2 # [ 0.422241] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1173worker1 # [ 0.423243] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1174client # [ 0.433252] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1175lb # [ 0.372313] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1176worker1 # [ 0.423826] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1177worker2 # [ 0.422289] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1178client # [ 0.433287] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1179lb # [ 0.372497] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1180worker1 # [ 0.423837] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1181client # [ 0.433309] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1182worker2 # [ 0.422337] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1183lb # [ 0.372513] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1184client # [ 0.433330] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1185lb # [ 0.372544] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1186worker1 # [ 0.423843] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1187worker2 # [ 0.422385] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1188client # [ 0.433352] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1189worker1 # [ 0.423889] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1190worker2 # [ 0.422432] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1191client # [ 0.433373] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1192client # [ 0.433395] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1193worker2 # [ 0.422478] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1194worker1 # [ 0.423937] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1195lb # [ 0.391418] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1196client # [ 0.433416] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1197worker1 # [ 0.423986] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1198worker2 # [ 0.422524] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1199lb # [ 0.391606] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1200client # [ 0.433442] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1201worker1 # [ 0.424036] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1202lb # [ 0.391623] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1203client # [ 0.433452] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1204lb # [ 0.391654] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1205worker1 # [ 0.424084] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1206client # [ 0.433457] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1207client # [ 0.434275] pci 0000:00:07.0: enabling device (0000 -> 0002)1208lb # [ 0.392112] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1209worker1 # [ 0.424133] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1210lb # [ 0.392298] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1211worker1 # [ 0.424194] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1212lb # [ 0.392315] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1213worker1 # [ 0.424243] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1214lb # [ 0.392347] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1215lb # [ 0.392367] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1216worker1 # [ 0.424290] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1217lb # [ 0.392826] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1218worker1 # [ 0.424360] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1219worker1 # [ 0.424407] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1220worker1 # [ 0.424429] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1221worker1 # [ 0.424450] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1222worker1 # [ 0.424472] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1223worker2 # [ 0.450702] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1224worker1 # [ 0.424494] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1225worker2 # [ 0.450766] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1226worker1 # [ 0.424516] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1227worker2 # [ 0.450789] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1228worker1 # [ 0.424538] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1229worker2 # [ 0.450811] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1230worker1 # [ 0.424564] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1231worker2 # [ 0.450835] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1232worker1 # [ 0.424587] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1233worker2 # [ 0.450858] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1234worker1 # [ 0.424609] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1235worker2 # [ 0.450880] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1236worker1 # [ 0.424632] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1237worker2 # [ 0.450902] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1238worker1 # [ 0.424655] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1239worker2 # [ 0.450928] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1240worker1 # [ 0.424677] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1241worker2 # [ 0.450952] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1242worker1 # [ 0.424698] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1243worker2 # [ 0.450975] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1244worker1 # [ 0.424720] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1245lb # [ 0.405083] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1246worker2 # [ 0.450999] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1247worker1 # [ 0.424741] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1248lb # [ 0.405129] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1249worker2 # [ 0.451020] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1250worker1 # [ 0.424763] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1251lb # [ 0.405632] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1252worker2 # [ 0.451041] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1253worker1 # [ 0.424785] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1254lb # [ 0.405825] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1255worker2 # [ 0.451062] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1256worker1 # [ 0.424811] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1257worker2 # [ 0.451083] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1258lb # [ 0.405856] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1259worker1 # [ 0.424821] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1260worker2 # [ 0.451104] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1261worker1 # [ 0.424826] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1262lb # [ 0.406280] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1263worker2 # [ 0.451125] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1264lb # [ 0.406462] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1265worker2 # [ 0.451146] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1266lb # [ 0.406709] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1267worker2 # [ 0.451175] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1268lb # [ 0.406896] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1269worker2 # [ 0.451184] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1270lb # [ 0.406927] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1271worker2 # [ 0.451189] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1272worker2 # [ 0.452004] pci 0000:00:07.0: enabling device (0000 -> 0002)1273lb # [ 0.407399] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1274client # [ 0.490834] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1275lb # [ 0.407586] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1276lb # [ 0.407616] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1277lb # [ 0.408064] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1278lb # [ 0.408252] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1279lb # [ 0.408284] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1280lb # [ 0.408747] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1281worker1 # [ 0.469760] pci 0000:00:07.0: enabling device (0000 -> 0002)1282lb # [ 0.409041] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1283lb # [ 0.409059] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1284lb # [ 0.409090] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1285lb # [ 0.409555] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1286lb # [ 0.409738] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1287lb # [ 0.409755] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1288lb # [ 0.409786] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1289lb # [ 0.410370] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1290client # [ 0.500890] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1291lb # [ 0.410382] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1292lb # [ 0.410387] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1293lb # [ 0.410433] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1294lb # [ 0.410482] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1295lb # [ 0.410530] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1296client # [ 0.505111] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1297lb # [ 0.410579] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1298client # [ 0.507292] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1299lb # [ 0.410627] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1300lb # [ 0.410675] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1301lb # [ 0.410722] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1302lb # [ 0.410769] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1303lb # [ 0.410816] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1304lb # [ 0.410885] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1305lb # [ 0.410933] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1306lb # [ 0.410956] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1307lb # [ 0.410978] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1308lb # [ 0.411000] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1309worker2 # [ 0.496769] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1310lb # [ 0.411023] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1311worker1 # [ 0.490779] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1312client # [ 0.518585] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1313lb # [ 0.411054] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1314worker1 # [ 0.493014] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1315lb # [ 0.411076] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1316lb # [ 0.411102] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1317lb # [ 0.411125] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1318lb # [ 0.411148] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1319lb # [ 0.411172] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1320lb # [ 0.411195] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1321lb # [ 0.411217] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1322lb # [ 0.411240] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1323lb # [ 0.411262] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1324worker2 # [ 0.499925] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1325client # [ 0.524730] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1326lb # [ 0.411284] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1327worker2 # [ 0.501982] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1328lb # [ 0.411306] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1329worker2 # [ 0.504049] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1330lb # [ 0.411328] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1331client # [ 0.526491] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1332lb # [ 0.411357] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1333worker2 # [ 0.506339] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1334worker1 # [ 0.504247] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1335lb # [ 0.411367] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1336lb # [ 0.411371] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1337lb # [ 0.412190] pci 0000:00:07.0: enabling device (0000 -> 0002)1338worker1 # [ 0.510298] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1339client # [ 0.536649] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1340worker1 # [ 0.512414] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1341client # [ 0.538654] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1342worker2 # [ 0.524334] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1343worker2 # [ 0.526150] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1344worker2 # [ 0.529640] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1345worker1 # [ 0.522915] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1346worker1 # [ 0.524782] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1347client # [ 0.549725] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1348client # [ 0.552963] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1349worker2 # [ 0.537297] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1350lb # [ 0.488535] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1351worker1 # [ 0.536094] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1352worker1 # [ 0.538942] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1353client # [ 0.566363] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1354worker2 # [ 0.543127] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1355worker1 # [ 0.540801] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1356worker2 # [ 0.545562] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1357client # [ 0.568968] msm_serial: driver initialized1358lb # [ 0.498369] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1359client # [ 0.569124] SuperH (H)SCI(F) driver initialized1360client # [ 0.569178] STM32 USART driver initialized1361worker1 # [ 0.550645] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1362worker2 # [ 0.558704] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1363lb # [ 0.509559] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1364lb # [ 0.511581] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1365worker1 # [ 0.562468] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1366lb # [ 0.513626] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1367lb # [ 0.515927] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1368worker2 # [ 0.567910] msm_serial: driver initialized1369worker1 # [ 0.565193] msm_serial: driver initialized1370worker2 # [ 0.568080] SuperH (H)SCI(F) driver initialized1371worker2 # [ 0.568132] STM32 USART driver initialized1372worker1 # [ 0.565942] SuperH (H)SCI(F) driver initialized1373worker1 # [ 0.565999] STM32 USART driver initialized1374lb # [ 0.526141] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1375lb # [ 0.528416] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1376client # [ 0.603788] loop: module loaded1377client # [ 0.603983] virtio_blk virtio2: 1/0/0 default/read/poll queues1378client # [ 0.604842] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1379lb # [ 0.539702] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1380lb # [ 0.542599] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1381worker2 # [ 0.597424] loop: module loaded1382worker2 # [ 0.597635] virtio_blk virtio2: 1/0/0 default/read/poll queues1383client # [ 0.612176] megasas: 07.734.00.00-rc11384worker2 # [ 0.598487] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1385client # [ 0.612939] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1386client # [ 0.615273] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001387client # [ 0.615302] Intel/Sharp Extended Query Table at 0x00311388client # [ 0.617038] Using buffer write method1389client # [ 0.617134] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1390worker1 # [ 0.598327] loop: module loaded1391worker1 # [ 0.598513] virtio_blk virtio2: 1/0/0 default/read/poll queues1392client # [ 0.618988] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001393client # [ 0.619015] Intel/Sharp Extended Query Table at 0x00311394worker1 # [ 0.599286] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1395lb # [ 0.553026] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1396lb # [ 0.559091] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1397worker2 # [ 0.606662] megasas: 07.734.00.00-rc11398worker2 # [ 0.607449] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399worker2 # [ 0.609354] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001400worker2 # [ 0.609379] Intel/Sharp Extended Query Table at 0x00311401client # [ 0.637555] Using buffer write method1402client # [ 0.637596] Concatenating MTD devices:1403client # [ 0.637600] (0): "0.flash"1404worker1 # [ 0.609941] megasas: 07.734.00.00-rc11405worker2 # [ 0.611033] Using buffer write method1406client # [ 0.637604] (1): "0.flash"1407lb # [ 0.566174] msm_serial: driver initialized1408client # [ 0.637613] into device "0.flash"1409worker1 # [ 0.610679] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1410lb # [ 0.566323] SuperH (H)SCI(F) driver initialized1411worker2 # [ 0.611100] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1412lb # [ 0.566379] STM32 USART driver initialized1413worker1 # [ 0.612800] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001414worker2 # [ 0.612878] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415worker1 # [ 0.612824] Intel/Sharp Extended Query Table at 0x00311416worker2 # [ 0.612900] Intel/Sharp Extended Query Table at 0x00311417worker2 # [ 0.614445] Using buffer write method1418worker2 # [ 0.614469] Concatenating MTD devices:1419worker2 # [ 0.614473] (0): "0.flash"1420worker2 # [ 0.614477] (1): "0.flash"1421worker2 # [ 0.614489] into device "0.flash"1422worker1 # [ 0.617592] Using buffer write method1423worker1 # [ 0.617657] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1424worker1 # [ 0.619307] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001425worker1 # [ 0.619328] Intel/Sharp Extended Query Table at 0x00311426worker1 # [ 0.621098] Using buffer write method1427worker1 # [ 0.621122] Concatenating MTD devices:1428worker1 # [ 0.621126] (0): "0.flash"1429worker1 # [ 0.621130] (1): "0.flash"1430worker1 # [ 0.621141] into device "0.flash"1431lb # [ 0.595716] loop: module loaded1432lb # [ 0.595893] virtio_blk virtio2: 1/0/0 default/read/poll queues1433lb # [ 0.596665] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1434lb # [ 0.609628] megasas: 07.734.00.00-rc11435lb # [ 0.610315] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1436lb # [ 0.612583] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001437lb # [ 0.612611] Intel/Sharp Extended Query Table at 0x00311438lb # [ 0.614163] Using buffer write method1439lb # [ 0.614240] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1440lb # [ 0.615929] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001441lb # [ 0.615953] Intel/Sharp Extended Query Table at 0x00311442lb # [ 0.633524] Using buffer write method1443lb # [ 0.633565] Concatenating MTD devices:1444lb # [ 0.633570] (0): "0.flash"1445lb # [ 0.633574] (1): "0.flash"1446lb # [ 0.633583] into device "0.flash"1447client # [ 0.901218] Freeing initrd memory: 26972K1448client # [ 0.907319] tun: Universal TUN/TAP device driver, 1.61449client # [ 0.911312] thunder_xcv, ver 1.01450client # [ 0.911362] thunder_bgx, ver 1.01451client # [ 0.911384] nicpf, ver 1.01452client # [ 0.913143] e1000: Intel(R) PRO/1000 Network Driver1453client # [ 0.913155] e1000: Copyright (c) 1999-2006 Intel Corporation.1454worker1 # [ 0.890059] Freeing initrd memory: 26976K1455client # [ 0.913181] e1000e: Intel(R) PRO/1000 Network Driver1456worker2 # [ 0.897364] Freeing initrd memory: 26944K1457client # [ 0.913189] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1458client # [ 0.913220] igb: Intel(R) Gigabit Ethernet Network Driver1459client # [ 0.913225] igb: Copyright (c) 2007-2014 Intel Corporation.1460client # [ 0.913249] igbvf: Intel(R) Gigabit Virtual Function Network Driver1461worker1 # [ 0.896021] tun: Universal TUN/TAP device driver, 1.61462client # [ 0.913254] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1463client # [ 0.913406] sky2: driver version 1.301464client # [ 0.915016] usbcore: registered new interface driver usb-storage1465worker2 # [ 0.903402] tun: Universal TUN/TAP device driver, 1.61466client # [ 0.915152] usbcore: registered new interface driver usbserial_generic1467client # [ 0.915166] usbserial: USB Serial support registered for generic1468client # [ 0.921013] ehci-pci 0000:00:07.0: EHCI Host Controller1469client # [ 0.921033] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11470worker1 # [ 0.899738] thunder_xcv, ver 1.01471client # [ 0.921306] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001472worker1 # [ 0.899781] thunder_bgx, ver 1.01473worker1 # [ 0.899802] nicpf, ver 1.01474worker1 # [ 0.900374] e1000: Intel(R) PRO/1000 Network Driver1475client # [ 0.927595] hv_vmbus: registering driver hyperv_keyboard1476worker1 # [ 0.900382] e1000: Copyright (c) 1999-2006 Intel Corporation.1477worker1 # [ 0.900403] e1000e: Intel(R) PRO/1000 Network Driver1478worker1 # [ 0.900411] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1479client # [ 0.929223] rtc-pl031 9010000.pl031: registered as rtc01480worker1 # [ 0.900439] igb: Intel(R) Gigabit Ethernet Network Driver1481worker1 # [ 0.900445] igb: Copyright (c) 2007-2014 Intel Corporation.1482worker2 # [ 0.907457] thunder_xcv, ver 1.01483client # [ 0.929252] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:31:35 UTC (1790076695)1484worker2 # [ 0.907502] thunder_bgx, ver 1.01485worker1 # [ 0.900466] igbvf: Intel(R) Gigabit Virtual Function Network Driver1486client # [ 0.929562] i2c_dev: i2c /dev entries driver1487worker2 # [ 0.907525] nicpf, ver 1.01488worker2 # [ 0.908081] e1000: Intel(R) PRO/1000 Network Driver1489worker1 # [ 0.900472] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1490worker1 # [ 0.900605] sky2: driver version 1.301491worker2 # [ 0.908090] e1000: Copyright (c) 1999-2006 Intel Corporation.1492client # [ 0.932614] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001493client # [ 0.932926] hub 1-0:1.0: USB hub found1494worker2 # [ 0.908113] e1000e: Intel(R) PRO/1000 Network Driver1495client # [ 0.932947] hub 1-0:1.0: 6 ports detected1496worker2 # [ 0.908122] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1497worker2 # [ 0.908152] igb: Intel(R) Gigabit Ethernet Network Driver1498worker1 # [ 0.909978] ehci-pci 0000:00:07.0: EHCI Host Controller1499worker2 # [ 0.908158] igb: Copyright (c) 2007-2014 Intel Corporation.1500worker1 # [ 0.910010] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11501worker2 # [ 0.908196] igbvf: Intel(R) Gigabit Virtual Function Network Driver1502worker1 # [ 0.910298] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001503worker2 # [ 0.908203] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1504worker2 # [ 0.908335] sky2: driver version 1.301505worker1 # [ 0.913031] usbcore: registered new interface driver usb-storage1506worker2 # [ 0.909962] usbcore: registered new interface driver usb-storage1507worker1 # [ 0.913120] usbcore: registered new interface driver usbserial_generic1508worker2 # [ 0.910054] usbcore: registered new interface driver usbserial_generic1509client # [ 0.937448] sdhci: Secure Digital Host Controller Interface driver1510worker1 # [ 0.913133] usbserial: USB Serial support registered for generic1511worker2 # [ 0.910068] usbserial: USB Serial support registered for generic1512client # [ 0.937459] sdhci: Copyright(c) Pierre Ossman1513worker2 # [ 0.911286] ehci-pci 0000:00:07.0: EHCI Host Controller1514client # [ 0.937721] Synopsys Designware Multimedia Card Interface Driver1515client # [ 0.938087] sdhci-pltfm: SDHCI platform and OF driver helper1516worker2 # [ 0.911316] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11517worker1 # [ 0.915950] hv_vmbus: registering driver hyperv_keyboard1518worker2 # [ 0.911602] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001519client # [ 0.942250] hid: raw HID events driver (C) Jiri Kosina1520worker1 # [ 0.917487] rtc-pl031 9010000.pl031: registered as rtc01521worker2 # [ 0.923267] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001522client # [ 0.942497] usbcore: registered new interface driver usbhid1523worker2 # [ 0.923584] hub 1-0:1.0: USB hub found1524client # [ 0.942504] usbhid: USB HID core driver1525worker2 # [ 0.923598] hub 1-0:1.0: 6 ports detected1526worker1 # [ 0.917512] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:31:35 UTC (1790076695)1527worker1 # [ 0.917806] i2c_dev: i2c /dev entries driver1528worker2 # [ 0.925726] hv_vmbus: registering driver hyperv_keyboard1529client # [ 0.945360] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1530worker2 # [ 0.927356] rtc-pl031 9010000.pl031: registered as rtc01531client # [ 0.946892] drop_monitor: Initializing network drop monitor service1532client # [ 0.947031] NET: Registered PF_INET6 protocol family1533worker2 # [ 0.927384] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:31:35 UTC (1790076695)1534worker2 # [ 0.927699] i2c_dev: i2c /dev entries driver1535client # [ 0.950138] Segment Routing with IPv61536worker1 # [ 0.923095] sdhci: Secure Digital Host Controller Interface driver1537client # [ 0.950158] In-situ OAM (IOAM) with IPv61538worker1 # [ 0.923106] sdhci: Copyright(c) Pierre Ossman1539client # [ 0.950186] NET: Registered PF_PACKET protocol family1540worker1 # [ 0.923364] Synopsys Designware Multimedia Card Interface Driver1541worker1 # [ 0.923722] sdhci-pltfm: SDHCI platform and OF driver helper1542worker1 # [ 0.925227] hid: raw HID events driver (C) Jiri Kosina1543worker1 # [ 0.928573] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001544worker2 # [ 0.932911] sdhci: Secure Digital Host Controller Interface driver1545worker1 # [ 0.929609] hub 1-0:1.0: USB hub found1546worker2 # [ 0.932924] sdhci: Copyright(c) Pierre Ossman1547worker1 # [ 0.930102] hub 1-0:1.0: 6 ports detected1548client # [ 0.951849] 9pnet: Installing 9P2000 support1549worker2 # [ 0.933186] Synopsys Designware Multimedia Card Interface Driver1550client # [ 0.951895] Key type dns_resolver registered1551worker2 # [ 0.933576] sdhci-pltfm: SDHCI platform and OF driver helper1552worker1 # [ 0.931196] usbcore: registered new interface driver usbhid1553worker1 # [ 0.931206] usbhid: USB HID core driver1554worker2 # [ 0.937841] hid: raw HID events driver (C) Jiri Kosina1555worker2 # [ 0.938083] usbcore: registered new interface driver usbhid1556client # [ 0.958859] registered taskstats version 11557worker2 # [ 0.938090] usbhid: USB HID core driver1558client # [ 0.959001] Loading compiled-in X.509 certificates1559worker1 # [ 0.933499] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1560worker1 # [ 0.935009] drop_monitor: Initializing network drop monitor service1561worker1 # [ 0.935171] NET: Registered PF_INET6 protocol family1562worker2 # [ 0.940959] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1563worker1 # [ 0.938226] Segment Routing with IPv61564worker2 # [ 0.942501] drop_monitor: Initializing network drop monitor service1565worker1 # [ 0.938243] In-situ OAM (IOAM) with IPv61566worker1 # [ 0.938270] NET: Registered PF_PACKET protocol family1567worker2 # [ 0.944555] NET: Registered PF_INET6 protocol family1568worker2 # [ 0.945744] Segment Routing with IPv61569lb # [ 0.894021] Freeing initrd memory: 26948K1570worker2 # [ 0.945775] In-situ OAM (IOAM) with IPv61571worker1 # [ 0.939902] 9pnet: Installing 9P2000 support1572worker2 # [ 0.945804] NET: Registered PF_PACKET protocol family1573worker1 # [ 0.939950] Key type dns_resolver registered1574client # [ 0.967652] Demotion targets for Node 0: null1575client # [ 0.967770] Key type .fscrypt registered1576client # [ 0.967776] Key type fscrypt-provisioning registered1577client # [ 0.967876] ima: No TPM chip found, activating TPM-bypass!1578worker2 # [ 0.947653] 9pnet: Installing 9P2000 support1579client # [ 0.967895] ima: Allocated hash algorithm: sha11580lb # [ 0.900201] tun: Universal TUN/TAP device driver, 1.61581worker2 # [ 0.947698] Key type dns_resolver registered1582client # [ 0.967916] ima: No architecture policies found1583client # [ 0.971964] input: gpio-keys as /devices/platform/gpio-keys/input/input01584worker2 # [ 0.954262] registered taskstats version 11585worker1 # [ 0.946704] registered taskstats version 11586worker2 # [ 0.954408] Loading compiled-in X.509 certificates1587worker1 # [ 0.946841] Loading compiled-in X.509 certificates1588lb # [ 0.903922] thunder_xcv, ver 1.01589lb # [ 0.903967] thunder_bgx, ver 1.01590lb # [ 0.903989] nicpf, ver 1.01591lb # [ 0.904530] e1000: Intel(R) PRO/1000 Network Driver1592lb # [ 0.904538] e1000: Copyright (c) 1999-2006 Intel Corporation.1593lb # [ 0.904561] e1000e: Intel(R) PRO/1000 Network Driver1594lb # [ 0.904568] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1595lb # [ 0.904597] igb: Intel(R) Gigabit Ethernet Network Driver1596lb # [ 0.904602] igb: Copyright (c) 2007-2014 Intel Corporation.1597lb # [ 0.904647] igbvf: Intel(R) Gigabit Virtual Function Network Driver1598lb # [ 0.904653] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1599worker1 # [ 0.955764] Demotion targets for Node 0: null1600lb # [ 0.904785] sky2: driver version 1.301601worker1 # [ 0.955874] Key type .fscrypt registered1602worker1 # [ 0.955880] Key type fscrypt-provisioning registered1603worker1 # [ 0.955985] ima: No TPM chip found, activating TPM-bypass!1604worker2 # [ 0.963069] Demotion targets for Node 0: null1605worker1 # [ 0.956004] ima: Allocated hash algorithm: sha11606worker2 # [ 0.963174] Key type .fscrypt registered1607worker1 # [ 0.956026] ima: No architecture policies found1608worker2 # [ 0.963181] Key type fscrypt-provisioning registered1609worker2 # [ 0.963276] ima: No TPM chip found, activating TPM-bypass!1610worker1 # [ 0.960050] input: gpio-keys as /devices/platform/gpio-keys/input/input01611worker2 # [ 0.963295] ima: Allocated hash algorithm: sha11612worker2 # [ 0.963315] ima: No architecture policies found1613lb # [ 0.913516] usbcore: registered new interface driver usb-storage1614lb # [ 0.913605] usbcore: registered new interface driver usbserial_generic1615worker2 # [ 0.967675] input: gpio-keys as /devices/platform/gpio-keys/input/input01616lb # [ 0.913619] usbserial: USB Serial support registered for generic1617lb # [ 0.914211] hv_vmbus: registering driver hyperv_keyboard1618lb # [ 0.915558] ehci-pci 0000:00:07.0: EHCI Host Controller1619lb # [ 0.915587] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11620lb # [ 0.915874] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001621client # [ 0.990075] clk: Disabling unused clocks1622client # [ 0.990102] PM: genpd: Disabling unused power domains1623lb # [ 0.921411] rtc-pl031 9010000.pl031: registered as rtc01624lb # [ 0.921440] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:31:35 UTC (1790076695)1625lb # [ 0.921798] i2c_dev: i2c /dev entries driver1626client # [ 0.994297] Freeing unused kernel memory: 4736K1627client # [ 0.994522] Run /init as init process1628lb # [ 0.927133] sdhci: Secure Digital Host Controller Interface driver1629lb # [ 0.927144] sdhci: Copyright(c) Pierre Ossman1630lb # [ 0.927407] Synopsys Designware Multimedia Card Interface Driver1631lb # [ 0.927773] sdhci-pltfm: SDHCI platform and OF driver helper1632lb # [ 0.931669] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001633lb # [ 0.932683] hub 1-0:1.0: USB hub found1634lb # [ 0.933204] hub 1-0:1.0: 6 ports detected1635worker1 # [ 0.977817] clk: Disabling unused clocks1636worker1 # [ 0.977845] PM: genpd: Disabling unused power domains1637lb # [ 0.934433] hid: raw HID events driver (C) Jiri Kosina1638worker2 # [ 0.986118] clk: Disabling unused clocks1639lb # [ 0.934674] usbcore: registered new interface driver usbhid1640worker2 # [ 0.986145] PM: genpd: Disabling unused power domains1641lb # [ 0.934680] usbhid: USB HID core driver1642worker1 # [ 0.981938] Freeing unused kernel memory: 4736K1643worker1 # [ 0.982141] Run /init as init process1644client # [ 1.009763] systemd[1]: Successfully made /usr/ read-only.1645lb # [ 0.937629] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1646worker2 # [ 0.990325] Freeing unused kernel memory: 4736K1647lb # [ 0.939186] drop_monitor: Initializing network drop monitor service1648worker2 # [ 0.991137] Run /init as init process1649lb # [ 0.939358] NET: Registered PF_INET6 protocol family1650lb # [ 0.942405] Segment Routing with IPv61651lb # [ 0.942426] In-situ OAM (IOAM) with IPv61652lb # [ 0.942454] NET: Registered PF_PACKET protocol family1653lb # [ 0.944062] 9pnet: Installing 9P2000 support1654lb # [ 0.944108] Key type dns_resolver registered1655lb # [ 0.950706] registered taskstats version 11656lb # [ 0.950846] Loading compiled-in X.509 certificates1657worker2 # [ 1.004828] systemd[1]: Successfully made /usr/ read-only.1658worker1 # [ 0.999601] systemd[1]: Successfully made /usr/ read-only.1659lb # [ 0.959973] Demotion targets for Node 0: null1660lb # [ 0.960096] Key type .fscrypt registered1661lb # [ 0.960103] Key type fscrypt-provisioning registered1662lb # [ 0.960201] ima: No TPM chip found, activating TPM-bypass!1663lb # [ 0.960221] ima: Allocated hash algorithm: sha11664lb # [ 0.960242] ima: No architecture policies found1665lb # [ 0.964372] input: gpio-keys as /devices/platform/gpio-keys/input/input01666lb # [ 0.982244] clk: Disabling unused clocks1667lb # [ 0.982270] PM: genpd: Disabling unused power domains1668lb # [ 0.986463] Freeing unused kernel memory: 4736K1669lb # [ 0.986670] Run /init as init process1670lb # [ 1.003152] systemd[1]: Successfully made /usr/ read-only.1671client # [ 1.179571] usb 1-1: new high-speed USB device number 2 using ehci-pci1672worker2 # [ 1.170600] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker1 # [ 1.177318] usb 1-1: new high-speed USB device number 2 using ehci-pci1674lb # [ 1.181012] usb 1-1: new high-speed USB device number 2 using ehci-pci1675client # [ 1.329504] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker2 # [ 1.323327] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677client # [ 1.344453] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678client # [ 1.350287] systemd[1]: Detected virtualization qemu.1679client # [ 1.351273] systemd[1]: Detected architecture arm64.1680client # [ 1.352270] systemd[1]: Running in initrd.1681client # [ 1.353620] systemd[1]: Initializing machine ID from random generator.1682client # [ 1.355031] systemd[1]: Hostname set to <client>.1683worker1 # [ 1.329883] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684worker2 # [ 1.339733] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1685worker2 # [ 1.352041] systemd[1]: Detected virtualization qemu.1686worker1 # [ 1.336233] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1687worker2 # [ 1.354090] systemd[1]: Detected architecture arm64.1688worker1 # [ 1.336294] systemd[1]: Detected virtualization qemu.1689worker2 # [ 1.356102] systemd[1]: Running in initrd.1690worker1 # [ 1.336369] systemd[1]: Detected architecture arm64.1691worker1 # [ 1.336394] systemd[1]: Running in initrd.1692worker2 # [ 1.358884] systemd[1]: Initializing machine ID from random generator.1693worker1 # [ 1.355374] systemd[1]: Initializing machine ID from random generator.1694worker2 # [ 1.361945] systemd[1]: Hostname set to <worker2>.1695worker1 # [ 1.358346] systemd[1]: Hostname set to <worker1>.1696lb # [ 1.333464] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11697client # [ 1.407799] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01698lb # [ 1.339430] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1699lb # [ 1.351813] systemd[1]: Detected virtualization qemu.1700lb # [ 1.353947] systemd[1]: Detected architecture arm64.1701lb # [ 1.355836] systemd[1]: Running in initrd.1702worker2 # [ 1.406866] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01703lb # [ 1.358470] systemd[1]: Initializing machine ID from random generator.1704lb # [ 1.361361] systemd[1]: Hostname set to <lb>.1705worker1 # [ 1.409482] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706lb # [ 1.417185] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707client # [ 1.531591] usb 1-2: new high-speed USB device number 3 using ehci-pci1708worker1 # [ 1.529311] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.534612] usb 1-2: new high-speed USB device number 3 using ehci-pci1710lb # [ 1.540964] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker1 # [ 1.619025] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712client # [ 1.667045] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713worker2 # [ 1.661890] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714client # [ 1.692268] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21715client # [ 1.697003] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01716worker1 # [ 1.679142] systemd[1]: Queued start job for default target Initrd Default Target.1717worker1 # [ 1.683201] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718worker1 # [ 1.684981] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01719worker1 # [ 1.688638] systemd[1]: Created slice Slice /system/modprobe.1720worker2 # [ 1.692880] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721worker1 # [ 1.689642] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1722worker2 # [ 1.695863] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723worker1 # [ 1.689675] systemd[1]: Expecting device /dev/disk/by-label/nixos...1724worker1 # [ 1.689701] systemd[1]: Reached target Path Units.1725worker1 # [ 1.689717] systemd[1]: Reached target Slice Units.1726worker1 # [ 1.689732] systemd[1]: Reached target Swaps.1727worker1 # [ 1.689746] systemd[1]: Reached target Timer Units.1728worker1 # [ 1.689889] systemd[1]: Listening on D-Bus System Message Bus Socket.1729worker1 # [ 1.690008] systemd[1]: Listening on Journal Socket (/dev/log).1730worker1 # [ 1.690122] systemd[1]: Listening on Journal Sockets.1731worker1 # [ 1.690223] systemd[1]: Listening on udev Control Socket.1732worker1 # [ 1.690308] systemd[1]: Listening on udev Kernel Socket.1733worker1 # [ 1.690324] systemd[1]: Reached target Socket Units.1734worker1 # [ 1.700064] systemd[1]: Starting Create List of Static Device Nodes...1735worker1 # [ 1.700134] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1736worker1 # [ 1.708839] systemd[1]: Mounting Kernel Configuration File System...1737lb # [ 1.671750] systemd[1]: bpf-restrict-fs: LSM BPF program attached1738worker1 # [ 1.717617] systemd[1]: Starting Journal Service...1739worker1 # [ 1.739558] systemd[1]: Starting Load Kernel Modules...1740worker1 # [ 1.739677] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1741worker2 # [ 1.748621] systemd[1]: Queued start job for default target Initrd Default Target.1742worker1 # [ 1.743380] systemd[1]: Starting Coldplug All udev Devices...1743worker2 # [ 1.756546] systemd[1]: Created slice Slice /system/modprobe.1744lb # [ 1.701775] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21745worker2 # [ 1.757614] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1746client # [ 1.778049] systemd[1]: Queued start job for default target Initrd Default Target.1747worker2 # [ 1.758684] systemd[1]: Expecting device /dev/disk/by-label/nixos...1748worker2 # [ 1.759509] systemd[1]: Reached target Path Units.1749worker2 # [ 1.760161] systemd[1]: Reached target Slice Units.1750worker2 # [ 1.760797] systemd[1]: Reached target Swaps.1751worker2 # [ 1.761374] systemd[1]: Reached target Timer Units.1752worker2 # [ 1.762177] systemd[1]: Listening on D-Bus System Message Bus Socket.1753lb # [ 1.708168] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01754worker2 # [ 1.763177] systemd[1]: Listening on Journal Socket (/dev/log).1755worker2 # [ 1.764044] systemd[1]: Listening on Journal Sockets.1756worker2 # [ 1.764801] systemd[1]: Listening on udev Control Socket.1757worker2 # [ 1.765653] systemd[1]: Listening on udev Kernel Socket.1758worker2 # [ 1.766433] systemd[1]: Reached target Socket Units.1759worker2 # [ 1.768446] systemd[1]: Starting Create List of Static Device Nodes...1760client # [ 1.789968] systemd[1]: Created slice Slice /system/modprobe.1761worker1 # [ 1.761420] systemd[1]: Finished Create List of Static Device Nodes.1762worker2 # [ 1.768512] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1763client # [ 1.791162] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1764client # [ 1.792553] systemd[1]: Expecting device /dev/disk/by-label/nixos...1765client # [ 1.793627] systemd[1]: Reached target Path Units.1766client # [ 1.794438] systemd[1]: Reached target Slice Units.1767client # [ 1.795261] systemd[1]: Reached target Swaps.1768client # [ 1.796075] systemd[1]: Reached target Timer Units.1769client # [ 1.797106] systemd[1]: Listening on D-Bus System Message Bus Socket.1770worker2 # [ 1.778204] systemd[1]: Mounting Kernel Configuration File System...1771client # [ 1.798427] systemd[1]: Listening on Journal Socket (/dev/log).1772client # [ 1.799632] systemd[1]: Listening on Journal Sockets.1773client # [ 1.800621] systemd[1]: Listening on udev Control Socket.1774client # [ 1.801704] systemd[1]: Listening on udev Kernel Socket.1775client # [ 1.802609] systemd[1]: Reached target Socket Units.1776client # [ 1.805100] systemd[1]: Starting Create List of Static Device Nodes...1777client # [ 1.806303] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1778worker1 # [ 1.782813] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1779worker1 # [ 1.783378] systemd[1]: Mounted Kernel Configuration File System.1780worker1 # [ 1.789677] systemd-journald[72]: Collecting audit messages is disabled.1781client # [ 1.815702] systemd[1]: Mounting Kernel Configuration File System...1782worker2 # [ 1.799661] systemd[1]: Starting Journal Service...1783worker1 # [ 1.793741] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1784worker1 # [ 1.795442] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1785worker2 # [ 1.801824] systemd[1]: Starting Load Kernel Modules...1786worker2 # [ 1.802711] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1787client # [ 1.823910] systemd[1]: Starting Journal Service...1788worker2 # [ 1.810030] systemd[1]: Starting Coldplug All udev Devices...1789worker1 # [ 1.806533] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01790worker1 # [ 1.806776] [drm] features: -virgl +edid -resource_blob -host_visible1791worker1 # [ 1.806786] [drm] features: -context_init1792worker1 # [ 1.807453] [drm] number of scanouts: 11793worker1 # [ 1.807471] [drm] number of cap sets: 01794worker2 # [ 1.825000] systemd[1]: Finished Create List of Static Device Nodes.1795worker2 # [ 1.825684] systemd[1]: Mounted Kernel Configuration File System.1796client # [ 1.847971] systemd[1]: Starting Load Kernel Modules...1797client # [ 1.848935] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1798worker1 # [ 1.828881] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1799worker2 # [ 1.835117] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1800worker1 # [ 1.828906] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01801lb # [ 1.790892] systemd[1]: Queued start job for default target Initrd Default Target.1802worker1 # [ 1.839816] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1803lb # [ 1.800275] systemd[1]: Created slice Slice /system/modprobe.1804client # [ 1.871330] systemd-journald[72]: Collecting audit messages is disabled.1805lb # [ 1.801575] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1806worker1 # [ 1.845692] systemd[1]: Starting Create Static Device Nodes in /dev...1807lb # [ 1.801620] systemd[1]: Expecting device /dev/disk/by-label/nixos...1808lb # [ 1.801655] systemd[1]: Reached target Path Units.1809lb # [ 1.801676] systemd[1]: Reached target Slice Units.1810lb # [ 1.801698] systemd[1]: Reached target Swaps.1811lb # [ 1.801718] systemd[1]: Reached target Timer Units.1812lb # [ 1.801928] systemd[1]: Listening on D-Bus System Message Bus Socket.1813lb # [ 1.802087] systemd[1]: Listening on Journal Socket (/dev/log).1814lb # [ 1.802232] systemd[1]: Listening on Journal Sockets.1815lb # [ 1.802368] systemd[1]: Listening on udev Control Socket.1816lb # [ 1.802484] systemd[1]: Listening on udev Kernel Socket.1817client # [ 1.880115] systemd[1]: Starting Coldplug All udev Devices...1818lb # [ 1.802508] systemd[1]: Reached target Socket Units.1819lb # [ 1.815151] systemd[1]: Starting Create List of Static Device Nodes...1820lb # [ 1.816336] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1821client # [ 1.891837] systemd[1]: Finished Create List of Static Device Nodes.1822client # [ 1.892837] systemd[1]: Mounted Kernel Configuration File System.1823worker1 # [ 1.861784] Console: switching to colour frame buffer device 160x501824lb # [ 1.825097] systemd[1]: Mounting Kernel Configuration File System...1825worker1 # [ 1.868527] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1826worker2 # [ 1.878507] systemd-journald[72]: Collecting audit messages is disabled.1827lb # [ 1.833199] systemd[1]: Starting Journal Service...1828client # [ 1.907974] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1829worker2 # [ 1.894706] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1830worker2 # [ 1.903206] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1831worker1 # [ 1.897862] systemd[1]: Finished Load Kernel Modules.1832worker2 # [ 1.905480] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1833worker1 # [ 1.901188] systemd[1]: Starting Apply Kernel Variables...1834worker2 # [ 1.907930] systemd[1]: Starting Create Static Device Nodes in /dev...1835lb # [ 1.861155] systemd[1]: Starting Load Kernel Modules...1836lb # [ 1.861284] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1837client # [ 1.931835] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1838worker1 # [ 1.914440] systemd[1]: Finished Create Static Device Nodes in /dev.1839worker2 # [ 1.919891] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01840worker2 # [ 1.920140] [drm] features: -virgl +edid -resource_blob -host_visible1841worker2 # [ 1.920150] [drm] features: -context_init1842worker2 # [ 1.920871] [drm] number of scanouts: 11843worker1 # [ 1.916640] systemd[1]: Reached target Preparation for Local File Systems.1844worker2 # [ 1.920888] [drm] number of cap sets: 01845worker1 # [ 1.916684] systemd[1]: Reached target Local File Systems.1846lb # [ 1.873552] systemd[1]: Starting Coldplug All udev Devices...1847client # [ 1.947616] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1848lb # [ 1.879645] systemd-journald[72]: Collecting audit messages is disabled.1849worker1 # [ 1.926883] systemd[1]: Starting Rule-based Manager for Device Events and Files...1850lb # [ 1.887324] systemd[1]: Finished Create List of Static Device Nodes.1851client # [ 1.956934] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01852worker2 # [ 1.934986] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1853client # [ 1.957184] [drm] features: -virgl +edid -resource_blob -host_visible1854worker2 # [ 1.935006] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01855client # [ 1.957193] [drm] features: -context_init1856lb # [ 1.889218] systemd[1]: Mounted Kernel Configuration File System.1857client # [ 1.957923] [drm] number of scanouts: 11858client # [ 1.957941] [drm] number of cap sets: 01859client # [ 1.972257] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1860client # [ 1.974787] systemd[1]: Starting Create Static Device Nodes in /dev...1861lb # [ 1.913117] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1862worker1 # [ 1.961583] systemd[1]: Finished Apply Kernel Variables.1863worker2 # [ 1.966836] Console: switching to colour frame buffer device 160x501864client # [ 1.988117] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1865client # [ 1.988137] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01866worker1 # [ 1.972820] systemd[1]: Started Journal Service.1867lb # [ 1.935691] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1868worker2 # [ 1.987262] systemd[1]: Finished Create Static Device Nodes in /dev.1869worker2 # [ 1.987471] systemd[1]: Reached target Preparation for Local File Systems.1870worker2 # [ 1.987495] systemd[1]: Reached target Local File Systems.1871worker2 # [ 1.991461] systemd[1]: Starting Rule-based Manager for Device Events and Files...1872worker1 # [ 1.976298] systemd-modules-load[73]: Inserted module 'dm_mod'1873worker1 # [ 1.977549] systemd-modules-load[73]: Module 'virtio_balloon' is built in1874worker1 # [ 1.978753] systemd-modules-load[73]: Module 'virtio_console' is built in1875worker1 # [ 1.979851] systemd-modules-load[73]: Inserted module 'virtio_gpu'1876client # [ 2.011937] Console: switching to colour frame buffer device 160x501877worker2 # [ 1.999365] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1878lb # [ 1.953626] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1879worker1 # [ 1.988805] systemd-modules-load[73]: Module 'virtio_rng' is built in1880lb # [ 1.956181] systemd[1]: Starting Create Static Device Nodes in /dev...1881worker1 # [ 1.989843] systemd[1]: Starting Create System Files and Directories...1882worker2 # [ 2.010171] systemd[1]: Finished Load Kernel Modules.1883client # [ 2.032208] systemd[1]: Finished Create Static Device Nodes in /dev.1884client # [ 2.032407] systemd[1]: Reached target Preparation for Local File Systems.1885client # [ 2.032430] systemd[1]: Reached target Local File Systems.1886worker2 # [ 2.015064] systemd[1]: Starting Apply Kernel Variables...1887lb # [ 1.963796] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1888client # [ 2.039986] systemd[1]: Starting Rule-based Manager for Device Events and Files...1889client # [ 2.045297] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1890worker1 # [ 2.018372] systemd[1]: Finished Create System Files and Directories.1891lb # [ 1.986171] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01892lb # [ 1.986416] [drm] features: -virgl +edid -resource_blob -host_visible1893lb # [ 1.986428] [drm] features: -context_init1894lb # [ 1.987152] [drm] number of scanouts: 11895lb # [ 1.987171] [drm] number of cap sets: 01896client # [ 2.061470] systemd[1]: Finished Load Kernel Modules.1897worker1 # [ 2.024710] systemd-udevd[79]: Using default interface naming scheme 'v261'.1898client # [ 2.056498] systemd-modules-load[74]: Inserted module 'dm_mod'1899client # [ 2.057792] systemd-modules-load[74]: Module 'virtio_balloon' is built in1900lb # [ 1.997280] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1901client # [ 2.058854] systemd-modules-load[74]: Module 'virtio_console' is built in1902lb # [ 1.997299] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01903client # [ 2.059890] systemd-modules-load[74]: Inserted module 'virtio_gpu'1904worker2 # [ 2.054728] systemd[1]: Started Journal Service.1905client # [ 2.077646] systemd[1]: Starting Apply Kernel Variables...1906client # [ 2.077826] systemd[1]: Started Journal Service.1907worker2 # [ 2.048396] systemd-modules-load[73]: Inserted module 'dm_mod'1908client # [ 2.072430] systemd-modules-load[74]: Module 'virtio_rng' is built in1909worker1 # [ 2.049597] systemd[1]: Started Rule-based Manager for Device Events and Files.1910worker2 # [ 2.053939] systemd-modules-load[73]: Module 'virtio_balloon' is built in1911lb # [ 2.017662] systemd[1]: Finished Create Static Device Nodes in /dev.1912worker2 # [ 2.055032] systemd-modules-load[73]: Module 'virtio_console' is built in1913lb # [ 2.017877] systemd[1]: Reached target Preparation for Local File Systems.1914lb # [ 2.017904] systemd[1]: Reached target Local File Systems.1915lb # [ 2.021979] systemd[1]: Starting Rule-based Manager for Device Events and Files...1916worker2 # [ 2.068238] systemd-modules-load[73]: Inserted module 'virtio_gpu'1917worker2 # [ 2.069333] systemd-modules-load[73]: Module 'virtio_rng' is built in1918worker2 # [ 2.071286] systemd[1]: Starting Create System Files and Directories...1919worker2 # [ 2.073402] systemd[1]: Finished Apply Kernel Variables.1920lb # [ 2.029412] Console: switching to colour frame buffer device 160x501921lb # [ 2.036311] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1922worker2 # [ 2.086422] systemd-udevd[78]: Using default interface naming scheme 'v261'.1923client # [ 2.117037] systemd[1]: Starting Create System Files and Directories...1924lb # [ 2.052525] systemd-modules-load[74]: Inserted module 'dm_mod'1925lb # [ 2.066604] systemd[1]: Finished Load Kernel Modules.1926lb # [ 2.066885] systemd[1]: Started Journal Service.1927lb # [ 2.057719] systemd-modules-load[74]: Module 'virtio_balloon' is built in1928lb # [ 2.058865] systemd-modules-load[74]: Module 'virtio_console' is built in1929lb # [ 2.059924] systemd-modules-load[74]: Inserted module 'virtio_gpu'1930worker1 # [ 2.108114] systemd[1]: Starting Virtual Console Setup...1931client # [ 2.132487] systemd-udevd[79]: Using default interface naming scheme 'v261'.1932worker2 # [ 2.117287] systemd[1]: Finished Create System Files and Directories.1933client # [ 2.137053] systemd[1]: Finished Apply Kernel Variables.1934lb # [ 2.072310] systemd-modules-load[74]: Module 'virtio_rng' is built in1935client # [ 2.139907] systemd[1]: Finished Create System Files and Directories.1936lb # [ 2.073372] systemd[1]: Starting Apply Kernel Variables...1937worker2 # [ 2.127494] systemd[1]: Started Rule-based Manager for Device Events and Files.1938lb # [ 2.087774] systemd[1]: Starting Create System Files and Directories...1939client # [ 2.162251] systemd[1]: Started Rule-based Manager for Device Events and Files.1940worker1 # [ 2.160560] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1941worker1 # [ 2.164177] systemd[1]: Finished Virtual Console Setup.1942lb # [ 2.128348] systemd-udevd[78]: Using default interface naming scheme 'v261'.1943lb # [ 2.131492] systemd[1]: Finished Apply Kernel Variables.1944worker2 # [ 2.183790] systemd[1]: Starting Virtual Console Setup...1945lb # [ 2.140363] systemd[1]: Finished Create System Files and Directories.1946client # [ 2.224123] systemd[1]: Starting Virtual Console Setup...1947lb # [ 2.161156] systemd[1]: Started Rule-based Manager for Device Events and Files.1948worker2 # [ 2.232449] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1949worker2 # [ 2.236054] systemd[1]: Finished Virtual Console Setup.1950client # [ 2.272543] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1951client # [ 2.276030] systemd[1]: Finished Virtual Console Setup.1952lb # [ 2.220252] systemd[1]: Starting Virtual Console Setup...1953lb # [ 2.268491] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1954lb # [ 2.271900] systemd[1]: Finished Virtual Console Setup.1955worker1 # [ 2.766488] systemd[1]: Finished Coldplug All udev Devices.1956worker1 # [ 2.768142] systemd[1]: Reached target System Initialization.1957worker1 # [ 2.772113] systemd[1]: Reached target Basic System.1958worker2 # [ 2.848732] systemd[1]: Finished Coldplug All udev Devices.1959worker2 # [ 2.849705] systemd[1]: Reached target System Initialization.1960worker2 # [ 2.850529] systemd[1]: Reached target Basic System.1961client # [ 2.874127] systemd[1]: Finished Coldplug All udev Devices.1962client # [ 2.875065] systemd[1]: Reached target System Initialization.1963client # [ 2.880124] systemd[1]: Reached target Basic System.1964worker1 # [ 2.925870] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1965lb # [ 2.882802] systemd[1]: Finished Coldplug All udev Devices.1966lb # [ 2.888141] systemd[1]: Reached target System Initialization.1967lb # [ 2.889394] systemd[1]: Reached target Basic System.1968worker1 # [ 2.948903] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1969worker1 # [ 2.951834] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line.1970worker2 # [ 2.996147] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1971client # [ 3.031919] (udev-worker)[105]: Network interface NamePolicy= disabled on kernel command line.1972worker2 # [ 3.022296] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973worker1 # [ 3.020805] systemd[1]: Found device /dev/disk/by-label/nixos.1974worker2 # [ 3.026854] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1975worker1 # [ 3.028263] systemd[1]: Reached target Initrd Root Device.1976worker1 # [ 3.033776] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1977client # [ 3.057559] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1978client # [ 3.061816] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1979worker1 # [ 3.083661] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1980worker1 # [ 3.090690] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1981worker2 # [ 3.096356] systemd[1]: Found device /dev/disk/by-label/nixos.1982lb # [ 3.046936] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1983worker1 # [ 3.095911] systemd[1]: Mounting /sysroot...1984client # [ 3.120100] systemd[1]: Found device /dev/disk/by-label/nixos.1985worker2 # [ 3.103312] systemd[1]: Reached target Initrd Root Device.1986client # [ 3.122790] systemd[1]: Reached target Initrd Root Device.1987lb # [ 3.058255] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1988client # [ 3.128211] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989worker2 # [ 3.109445] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1990lb # [ 3.065426] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1991worker1 # [ 3.142156] EXT4-fs (vda): mounted filesystem 9c69db1f-c800-4716-bd16-1a58065274da r/w with ordered data mode. Quota mode: none.1992worker1 # [ 3.134845] systemd[1]: Mounted /sysroot.1993worker1 # [ 3.136438] systemd[1]: Reached target Initrd Root File System.1994worker1 # [ 3.140435] systemd[1]: Starting Mountpoints Configured in the Real Root...1995worker2 # [ 3.158207] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1996worker2 # [ 3.163735] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1997client # [ 3.183970] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1998client # [ 3.189161] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1999worker2 # [ 3.172304] systemd[1]: Mounting /sysroot...2000worker1 # [ 3.168260] systemd-sysroot-fstab-check[117]: /sysroot should be mounted in the initrd, will request daemon-reload.
loading 2,323 hidden lines…
4324lb # [ 52.636874] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4325worker1 # [ 52.683042] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.277Z level=INFO msg="Uploading 1 narinfos"4326lb # [ 52.646082] niks3-server[1033]: 2026/09/22 11:32:27 INFO Signed narinfos id=4 count=14327lb # [ 52.655900] niks3-server[1033]: 2026/09/22 11:32:27 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4328lb # [ 52.673085] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4329lb # [ 52.687678] niks3-server[1033]: 2026/09/22 11:32:27 INFO Completed upload id=44330worker1 # [ 52.732950] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.328Z level=INFO msg="Upload complete. (403ms)"4331lb # [ 52.691291] niks3-server[1033]: 2026/09/22 11:32:27 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4332worker1 # [ 52.764539] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14333worker1 # [ 52.780786] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04334worker1 # [ 52.793476] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14335worker1 # [ 52.803869] nix-daemon[852]: reaped child process 927, status = succeeded4336worker1 # [ 52.813727] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404337worker1 # [ 52.829096] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404338worker1 # [ 52.850508] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.445Z level=INFO msg="All 2 paths already cached"4339client # copying 1 paths to 'lb:50051'...4340worker1 # [ 52.869298] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430404341client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4342worker1 # [ 52.900176] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4343worker1 # [ 53.142188] nix-daemon[852]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4344worker1 # [ 53.169357] nix-daemon[852]: reaped child process 957, status = succeeded4345worker1 # [ 53.321436] nix-daemon[852]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4346worker1 # [ 53.348198] nix-daemon[852]: reaped child process 962, status = succeeded4347lb # [ 53.300714] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received uploads request method=POST path=/api/pending_closures4348worker1 # [ 53.362458] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.957Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4349worker1 # [ 53.364141] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.957Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4350lb # [ 53.357008] niks3-server[1033]: 2026/09/22 11:32:27 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4351worker1 # [ 53.432896] nix-grpc-daemon[867]: time=2026-09-22T11:32:28.027Z level=INFO msg="Uploading 1 narinfos"4352lb # [ 53.379158] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4353lb # [ 53.393467] niks3-server[1033]: 2026/09/22 11:32:27 INFO Signed narinfos id=5 count=14354lb # [ 53.410586] niks3-server[1033]: 2026/09/22 11:32:28 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4355lb # [ 53.422421] niks3-server[1033]: 2026/09/22 11:32:28 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4356worker1 # [ 53.474184] nix-grpc-daemon[867]: time=2026-09-22T11:32:28.069Z level=INFO msg="Upload complete. (543ms)"4357lb # [ 53.430618] niks3-server[1033]: 2026/09/22 11:32:28 INFO Completed upload id=54358lb # [ 53.435851] niks3-server[1033]: 2026/09/22 11:32:28 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4359worker1 # [ 53.488415] nix-grpc-daemon[768]: ts=2026-09-22T11:32:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=7524360worker1 # [ 53.505190] nix-grpc-daemon[768]: ts=2026-09-22T11:32:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404361worker1 # [ 53.527154] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4362worker1 # [ 53.538038] nix-daemon[852]: reaped child process 952, status = succeeded4363client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4364client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4365worker1 # [ 53.557821] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4366client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4367client # farm-top> LOG-farm-top-t14368worker1 # [ 54.424703] nix-daemon[852]: reaped child process 966, status = succeeded4369worker1 # [ 54.556441] nix-daemon[852]: accepted connection from pid 982, user nix-grpc-daemon (trusted)4370worker1 # [ 54.574665] nix-daemon[852]: reaped child process 985, status = succeeded4371lb # [ 54.535565] niks3-server[1033]: 2026/09/22 11:32:29 INFO Received uploads request method=POST path=/api/pending_closures4372worker1 # [ 54.592486] nix-grpc-daemon[867]: time=2026-09-22T11:32:29.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4373worker1 # [ 54.594381] nix-grpc-daemon[867]: time=2026-09-22T11:32:29.186Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4374lb # [ 54.577610] niks3-server[1033]: 2026/09/22 11:32:29 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4375lb # [ 54.593063] niks3-server[1033]: 2026/09/22 11:32:29 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4376lb # [ 54.604061] niks3-server[1033]: 2026/09/22 11:32:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4377worker1 # [ 54.653893] nix-grpc-daemon[867]: time=2026-09-22T11:32:29.248Z level=INFO msg="Uploading 1 narinfos"4378lb # [ 54.612987] niks3-server[1033]: 2026/09/22 11:32:29 INFO Signed narinfos id=6 count=14379lb # [ 54.618699] niks3-server[1033]: 2026/09/22 11:32:29 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4380lb # [ 54.629784] niks3-server[1033]: 2026/09/22 11:32:29 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4381lb # [ 54.636584] niks3-server[1033]: 2026/09/22 11:32:29 INFO Completed upload id=64382worker1 # [ 54.681151] nix-grpc-daemon[867]: time=2026-09-22T11:32:29.276Z level=INFO msg="Upload complete. (305ms)"4383lb # [ 54.640543] niks3-server[1033]: 2026/09/22 11:32:29 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4384worker1 # [ 54.697088] nix-grpc-daemon[768]: ts=2026-09-22T11:32:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14385worker1 # [ 54.717022] nix-daemon[852]: reaped child process 964, status = succeeded4386client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 7.46 seconds)4387client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4388client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds)4389client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4390client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4391client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4392client # copying 3 paths...4393client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4394client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4395client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4396client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)4397(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.02 seconds)4398subtest: repeat is answered Cached by the scheduler without building4399worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4400worker1 # finding garbage collector roots...4401worker1 # deleting specified paths...4402worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4403worker1 # deleting unused links...4404worker1 # note: hard linking is currently saving 0.0 KiB4405worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4406worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4407worker2 # finding garbage collector roots...4408worker2 # deleting specified paths...4409worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4410worker2 # deleting unused links...4411worker2 # note: hard linking is currently saving 0.0 KiB4412worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.25 seconds)4413client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24414worker2 # [ 56.010446] nix-grpc-daemon[771]: ts=2026-09-22T11:32:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457244415client # don't know how to build these paths:4416client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4417worker2 # [ 56.025593] nix-grpc-daemon[771]: ts=2026-09-22T11:32:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457244418worker1 # [ 56.053983] nix-grpc-daemon[768]: ts=2026-09-22T11:32:30Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14419client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.30 seconds)4420client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4421client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds)4422lb: must succeed: curl -sf localhost:9901/clusters4423lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4424lb: must succeed: curl -sf localhost:9901/clusters4425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4426worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4427worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4428(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.21 seconds)4429subtest: inputs already in the cache are not sent to the scheduler4430lb: must succeed: curl -sf localhost:9901/clusters4431lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4432lb: must succeed: curl -sf localhost:9901/clusters4433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4434worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4435worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4436client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24437worker2 # [ 56.914257] nix-grpc-daemon[771]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457244438client # don't know how to build these paths:4439client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4440worker1 # [ 56.920854] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404441worker1 # [ 56.931724] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.526Z level=INFO msg="All 2 paths already cached"4442worker1 # [ 56.947915] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04443worker1 # [ 56.956149] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14444worker1 # [ 56.993145] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404445worker1 # [ 57.009119] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404446worker1 # [ 57.023803] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.619Z level=INFO msg="All 2 paths already cached"4447client # copying 1 paths to 'lb:50051'...4448client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4449worker1 # [ 57.035372] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430404450worker1 # [ 57.050299] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4451worker1 # [ 57.203731] nix-daemon[852]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4452worker1 # [ 57.229673] nix-daemon[852]: reaped child process 1017, status = succeeded4453worker1 # [ 57.322493] nix-daemon[852]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4454worker1 # [ 57.340972] nix-daemon[852]: reaped child process 1022, status = succeeded4455lb # [ 57.295002] niks3-server[1033]: 2026/09/22 11:32:31 INFO Received uploads request method=POST path=/api/pending_closures4456worker1 # [ 57.349461] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.944Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4457worker1 # [ 57.350940] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.944Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4458lb # [ 57.324683] niks3-server[1033]: 2026/09/22 11:32:31 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4459worker1 # [ 57.384467] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.979Z level=INFO msg="Uploading 1 narinfos"4460lb # [ 57.339223] niks3-server[1033]: 2026/09/22 11:32:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4461lb # [ 57.351135] niks3-server[1033]: 2026/09/22 11:32:31 INFO Signed narinfos id=7 count=14462lb # [ 57.359688] niks3-server[1033]: 2026/09/22 11:32:31 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4463lb # [ 57.373747] niks3-server[1033]: 2026/09/22 11:32:31 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4464worker1 # [ 57.425058] nix-grpc-daemon[867]: time=2026-09-22T11:32:32.020Z level=INFO msg="Upload complete. (357ms)"4465lb # [ 57.381156] niks3-server[1033]: 2026/09/22 11:32:31 INFO Completed upload id=74466lb # [ 57.385444] niks3-server[1033]: 2026/09/22 11:32:31 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4467worker1 # [ 57.432790] nix-grpc-daemon[768]: ts=2026-09-22T11:32:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=7524468worker1 # [ 57.447066] nix-grpc-daemon[768]: ts=2026-09-22T11:32:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404469worker1 # [ 57.454820] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4470worker1 # [ 57.460007] nix-daemon[852]: reaped child process 1012, status = succeeded4471client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4472worker1 # [ 57.474917] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4473client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4474client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4475client # farm-top> LOG-farm-top-t1b4476lb # [ 58.325833] envoy[893]: conn peer=[2001:db8:1::1]:36304 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9579 tx=4449 ms=72844477worker1 # [ 58.579431] nix-daemon[852]: reaped child process 1026, status = succeeded4478worker1 # [ 58.651743] nix-daemon[852]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)4479worker1 # [ 58.674659] nix-daemon[852]: reaped child process 1045, status = succeeded4480lb # [ 58.330764] envoy[893]: conn peer=[2001:db8:1::1]:40624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=481 ms=704481lb # [ 58.628334] niks3-server[1033]: 2026/09/22 11:32:33 INFO Received uploads request method=POST path=/api/pending_closures4482worker1 # [ 58.687679] nix-grpc-daemon[867]: time=2026-09-22T11:32:33.282Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4483worker1 # [ 58.693343] nix-grpc-daemon[867]: time=2026-09-22T11:32:33.285Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4484lb # [ 58.672421] niks3-server[1033]: 2026/09/22 11:32:33 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4485lb # [ 58.690798] niks3-server[1033]: 2026/09/22 11:32:33 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4486worker1 # [ 58.747306] nix-grpc-daemon[867]: time=2026-09-22T11:32:33.342Z level=INFO msg="Uploading 1 narinfos"4487lb # [ 58.703238] niks3-server[1033]: 2026/09/22 11:32:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4488lb # [ 58.712060] niks3-server[1033]: 2026/09/22 11:32:33 INFO Signed narinfos id=8 count=14489lb # [ 58.714052] niks3-server[1033]: 2026/09/22 11:32:33 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4490lb # [ 58.725102] niks3-server[1033]: 2026/09/22 11:32:33 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4491lb # [ 58.729123] niks3-server[1033]: 2026/09/22 11:32:33 INFO Completed upload id=84492lb # [ 58.731314] niks3-server[1033]: 2026/09/22 11:32:33 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4493worker1 # [ 58.774131] nix-grpc-daemon[867]: time=2026-09-22T11:32:33.369Z level=INFO msg="Upload complete. (289ms)"4494worker1 # [ 58.789308] nix-grpc-daemon[768]: ts=2026-09-22T11:32:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14495worker1 # [ 58.805993] nix-daemon[852]: reaped child process 1024, status = succeeded4496client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.13 seconds)4497client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4498client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4499lb: must succeed: curl -sf localhost:9901/clusters4500lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4501lb: must succeed: curl -sf localhost:9901/clusters4502lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4503worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4504worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4505(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.65 seconds)4506subtest: outputs read back through any worker carry the cache signature4507client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24508worker2 # [ 59.391788] nix-grpc-daemon[771]: ts=2026-09-22T11:32:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457244509client # don't know how to build these paths:4510client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4511worker1 # [ 59.399389] nix-grpc-daemon[768]: ts=2026-09-22T11:32:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404512worker1 # [ 59.422950] nix-grpc-daemon[768]: ts=2026-09-22T11:32:34Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04513worker2 # [ 59.431823] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244514worker1 # [ 59.429111] nix-grpc-daemon[768]: ts=2026-09-22T11:32:34Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14515worker2 # [ 59.448382] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457244516client # copying 1 paths to 'lb:50051'...4517client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4518worker2 # [ 59.464708] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457244519worker2 # [ 59.481314] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4520worker2 # [ 59.510515] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=info event=niks3_push_started pid=8674521worker2 # [ 59.674520] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.261Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4522worker2 # [ 59.785949] nix-daemon[842]: accepted connection from pid 880, user nix-grpc-daemon (trusted)4523worker2 # [ 59.809716] nix-daemon[842]: reaped child process 883, status = succeeded4524worker2 # [ 59.910545] nix-daemon[842]: accepted connection from pid 885, user nix-grpc-daemon (trusted)4525worker2 # [ 59.927797] nix-daemon[842]: reaped child process 888, status = succeeded4526lb # [ 59.885479] niks3-server[1033]: 2026/09/22 11:32:34 INFO Received uploads request method=POST path=/api/pending_closures4527worker2 # [ 59.943654] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.530Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4528worker2 # [ 59.947478] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.532Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4529lb # [ 59.919300] niks3-server[1033]: 2026/09/22 11:32:34 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4530lb # [ 59.932641] niks3-server[1033]: 2026/09/22 11:32:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4531worker2 # [ 59.988190] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.574Z level=INFO msg="Uploading 1 narinfos"4532lb # [ 59.945822] niks3-server[1033]: 2026/09/22 11:32:34 INFO Signed narinfos id=9 count=14533lb # [ 59.953170] niks3-server[1033]: 2026/09/22 11:32:34 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4534lb # [ 59.967399] niks3-server[1033]: 2026/09/22 11:32:34 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4535worker2 # [ 60.024809] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.611Z level=INFO msg="Upload complete. (346ms)"4536lb # [ 59.978797] niks3-server[1033]: 2026/09/22 11:32:34 INFO Completed upload id=94537lb # [ 59.982610] niks3-server[1033]: 2026/09/22 11:32:34 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4538worker2 # [ 60.032320] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=5124539worker2 # [ 60.048330] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244540worker2 # [ 60.056628] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4541worker2 # [ 60.061669] nix-daemon[842]: reaped child process 865, status = succeeded4542client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4543client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4544worker2 # [ 60.075352] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4545client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4546client # farm-a> LOG-farm-a-sig4547worker2 # [ 61.185137] nix-daemon[842]: reaped child process 892, status = succeeded4548worker2 # [ 61.262982] nix-daemon[842]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4549worker2 # [ 61.281760] nix-daemon[842]: reaped child process 911, status = succeeded4550lb # [ 61.239295] niks3-server[1033]: 2026/09/22 11:32:35 INFO Received uploads request method=POST path=/api/pending_closures4551worker2 # [ 61.307344] nix-grpc-daemon[867]: time=2026-09-22T11:32:35.890Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4552worker2 # [ 61.310573] nix-grpc-daemon[867]: time=2026-09-22T11:32:35.890Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4553lb # [ 61.293707] niks3-server[1033]: 2026/09/22 11:32:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4554lb # [ 61.308752] niks3-server[1033]: 2026/09/22 11:32:35 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4555lb # [ 61.316220] niks3-server[1033]: 2026/09/22 11:32:35 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4556lb # [ 61.325981] niks3-server[1033]: 2026/09/22 11:32:35 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4557lb # [ 61.333258] niks3-server[1033]: 2026/09/22 11:32:35 INFO Signed narinfos id=10 count=14558worker2 # [ 61.383142] nix-grpc-daemon[867]: time=2026-09-22T11:32:35.968Z level=INFO msg="Uploading 1 narinfos"4559lb # [ 61.348439] niks3-server[1033]: 2026/09/22 11:32:35 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4560lb # [ 61.356160] niks3-server[1033]: 2026/09/22 11:32:35 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4561lb # [ 61.362009] niks3-server[1033]: 2026/09/22 11:32:35 INFO Completed upload id=104562worker2 # [ 61.411291] nix-grpc-daemon[867]: time=2026-09-22T11:32:35.998Z level=INFO msg="Upload complete. (324ms)"4563worker2 # [ 61.423384] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14564worker1 # [ 61.424805] nix-grpc-daemon[768]: ts=2026-09-22T11:32:36Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04565worker1 # [ 61.425117] nix-grpc-daemon[768]: ts=2026-09-22T11:32:36Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14566worker2 # [ 61.444757] nix-daemon[842]: reaped child process 890, status = succeeded4567worker2 # [ 61.473181] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244568worker2 # [ 61.489457] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457244569worker2 # [ 61.504416] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.091Z level=INFO msg="All 1 paths already cached"4570client # copying 1 paths to 'lb:50051'...4571client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4572worker2 # [ 61.515032] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457244573worker2 # [ 61.529305] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4574worker2 # [ 61.683224] nix-daemon[842]: accepted connection from pid 915, user nix-grpc-daemon (trusted)4575worker2 # [ 61.706262] nix-daemon[842]: reaped child process 918, status = succeeded4576worker2 # [ 61.805699] nix-daemon[842]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4577worker2 # [ 61.823912] nix-daemon[842]: reaped child process 923, status = succeeded4578lb # [ 61.777311] niks3-server[1033]: 2026/09/22 11:32:36 INFO Received uploads request method=POST path=/api/pending_closures4579worker2 # [ 61.838984] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.425Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4580worker2 # [ 61.840580] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.425Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4581worker2 # [ 61.876758] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.463Z level=INFO msg="Uploading 1 narinfos"4582lb # [ 61.818057] niks3-server[1033]: 2026/09/22 11:32:36 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4583lb # [ 61.834808] niks3-server[1033]: 2026/09/22 11:32:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4584lb # [ 61.844155] niks3-server[1033]: 2026/09/22 11:32:36 INFO Signed narinfos id=11 count=14585lb # [ 61.849716] niks3-server[1033]: 2026/09/22 11:32:36 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4586lb # [ 61.858760] niks3-server[1033]: 2026/09/22 11:32:36 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4587lb # [ 61.866293] niks3-server[1033]: 2026/09/22 11:32:36 INFO Completed upload id=114588worker2 # [ 61.914801] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.501Z level=INFO msg="Upload complete. (369ms)"4589lb # [ 61.869142] niks3-server[1033]: 2026/09/22 11:32:36 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4590worker2 # [ 61.924286] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=6244591worker2 # [ 61.939793] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244592worker2 # [ 61.946601] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4593worker2 # [ 61.951657] nix-daemon[842]: reaped child process 913, status = succeeded4594client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4595worker2 # [ 61.965279] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4596client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4597client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4598client # farm-b> LOG-farm-b-sig4599worker2 # [ 63.401200] nix-daemon[842]: reaped child process 927, status = succeeded4600worker2 # [ 63.491351] nix-daemon[842]: accepted connection from pid 943, user nix-grpc-daemon (trusted)4601worker2 # [ 63.510448] nix-daemon[842]: reaped child process 946, status = succeeded4602lb # [ 63.467038] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received uploads request method=POST path=/api/pending_closures4603worker2 # [ 63.525928] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.112Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4604worker2 # [ 63.527444] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.112Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4605lb # [ 63.503098] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4606lb # [ 63.518130] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4607worker2 # [ 63.572576] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.159Z level=INFO msg="Uploading 1 narinfos"4608lb # [ 63.529472] niks3-server[1033]: 2026/09/22 11:32:38 INFO Signed narinfos id=12 count=14609lb # [ 63.535636] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4610lb # [ 63.546936] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4611worker2 # [ 63.601851] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.188Z level=INFO msg="Upload complete. (226ms)"4612lb # [ 63.553729] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4613lb # [ 63.557341] niks3-server[1033]: 2026/09/22 11:32:38 INFO Completed upload id=124614lb # [ 63.561570] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4615worker1 # [ 63.618251] nix-grpc-daemon[768]: ts=2026-09-22T11:32:38Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04616worker2 # [ 63.613089] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14617worker2 # [ 63.631325] nix-daemon[842]: reaped child process 925, status = succeeded4618worker1 # [ 63.624212] nix-grpc-daemon[768]: ts=2026-09-22T11:32:38Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14619worker2 # [ 63.669204] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244620worker2 # [ 63.686202] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457244621worker2 # [ 63.700531] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.287Z level=INFO msg="All 2 paths already cached"4622client # copying 1 paths to 'lb:50051'...4623client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4624worker2 # [ 63.711628] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457244625worker2 # [ 63.725130] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4626worker2 # [ 63.887315] nix-daemon[842]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4627worker2 # [ 63.911386] nix-daemon[842]: reaped child process 953, status = succeeded4628worker2 # [ 64.006351] nix-daemon[842]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4629worker2 # [ 64.025163] nix-daemon[842]: reaped child process 958, status = succeeded4630lb # [ 63.980859] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received uploads request method=POST path=/api/pending_closures4631worker2 # [ 64.038596] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.625Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4632worker2 # [ 64.040156] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.625Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4633lb # [ 64.007587] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4634worker2 # [ 64.068090] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.655Z level=INFO msg="Uploading 1 narinfos"4635lb # [ 64.019620] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4636lb # [ 64.027309] niks3-server[1033]: 2026/09/22 11:32:38 INFO Signed narinfos id=13 count=14637lb # [ 64.033409] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4638lb # [ 64.044581] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4639lb # [ 64.052672] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4640worker2 # [ 64.105369] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.692Z level=INFO msg="Upload complete. (363ms)"4641lb # [ 64.057602] niks3-server[1033]: 2026/09/22 11:32:38 INFO Completed upload id=134642worker2 # [ 64.112353] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=7524643worker2 # [ 64.127442] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244644worker2 # [ 64.134016] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4645worker2 # [ 64.139048] nix-daemon[842]: reaped child process 948, status = succeeded4646client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4647worker2 # [ 64.152197] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4648client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4649client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4650client # farm-top> LOG-farm-top-sig4651worker2 # [ 65.387908] nix-daemon[842]: reaped child process 962, status = succeeded4652worker2 # [ 65.495289] nix-daemon[842]: accepted connection from pid 978, user nix-grpc-daemon (trusted)4653worker2 # [ 65.518410] nix-daemon[842]: reaped child process 981, status = succeeded4654lb # [ 65.470668] niks3-server[1033]: 2026/09/22 11:32:40 INFO Received uploads request method=POST path=/api/pending_closures4655worker2 # [ 65.568698] nix-grpc-daemon[867]: time=2026-09-22T11:32:40.154Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4656worker2 # [ 65.571460] nix-grpc-daemon[867]: time=2026-09-22T11:32:40.154Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4657lb # [ 65.554948] niks3-server[1033]: 2026/09/22 11:32:40 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4658worker2 # [ 65.626516] nix-grpc-daemon[867]: time=2026-09-22T11:32:40.213Z level=INFO msg="Uploading 1 narinfos"4659lb # [ 65.571810] niks3-server[1033]: 2026/09/22 11:32:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4660lb # [ 65.582691] niks3-server[1033]: 2026/09/22 11:32:40 INFO Signed narinfos id=14 count=14661lb # [ 65.588240] niks3-server[1033]: 2026/09/22 11:32:40 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4662lb # [ 65.596941] niks3-server[1033]: 2026/09/22 11:32:40 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4663lb # [ 65.607239] niks3-server[1033]: 2026/09/22 11:32:40 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4664lb # [ 65.612259] niks3-server[1033]: 2026/09/22 11:32:40 INFO Completed upload id=144665worker2 # [ 65.660644] nix-grpc-daemon[867]: time=2026-09-22T11:32:40.247Z level=INFO msg="Upload complete. (298ms)"4666lb # [ 65.614853] niks3-server[1033]: 2026/09/22 11:32:40 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4667worker2 # [ 65.673289] nix-grpc-daemon[771]: ts=2026-09-22T11:32:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14668worker2 # [ 65.692875] nix-daemon[842]: reaped child process 960, status = succeeded4669client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.54 seconds)4670client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4671client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.18 seconds)4672client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4673worker1 # [ 66.026878] nix-grpc-daemon[768]: ts=2026-09-22T11:32:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:408464674client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4675client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4676client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4677client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4678worker1 # [ 66.057850] nix-grpc-daemon[768]: ts=2026-09-22T11:32:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:408464679worker1 # [ 66.066801] nix-grpc-daemon[768]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4680worker1 # [ 66.084780] nix-grpc-daemon[768]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4681worker1 # [ 66.099120] nix-grpc-daemon[768]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4682client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds)4683client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4684worker2 # [ 66.332682] nix-grpc-daemon[771]: ts=2026-09-22T11:32:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:525304685worker2 # [ 66.375078] nix-grpc-daemon[771]: ts=2026-09-22T11:32:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:525304686client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.26 seconds)4687client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4688client # finding garbage collector roots...4689client # removing stale temporary roots file "/nix/var/nix/temproots/882"4690client # removing stale temporary roots file "/nix/var/nix/temproots/846"4691client # removing stale temporary roots file "/nix/var/nix/temproots/859"4692client # removing stale temporary roots file "/nix/var/nix/temproots/805"4693client # removing stale temporary roots file "/nix/var/nix/temproots/867"4694client # removing stale temporary roots file "/nix/var/nix/temproots/826"4695client # removing stale temporary roots file "/nix/var/nix/temproots/790"4696client # removing stale temporary roots file "/nix/var/nix/temproots/838"4697client # deleting specified paths...4698client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4699client # deleting unused links...4700client # note: hard linking is currently saving 0.0 KiB4701worker2 # [ 66.728645] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457244702worker1 # [ 66.734594] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404703worker1 # [ 66.747803] nix-grpc-daemon[867]: time=2026-09-22T11:32:41.342Z level=INFO msg="All 1 paths already cached"4704worker2 # [ 66.760570] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:457244705worker1 # [ 66.765793] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:430404706client # copying 3 paths...4707worker2 # [ 66.784845] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:457244708client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4709worker2 # [ 66.813018] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:457244710worker2 # [ 66.820425] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=2324711worker1 # [ 66.824137] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:430404712worker1 # [ 66.831980] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_out=1764713worker1 # [ 66.840307] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:430404714worker1 # [ 66.846900] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_out=1284715client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4716client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4717client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.55 seconds)4718(finished: subtest: outputs read back through any worker carry the cache signature, in 7.77 seconds)4719subtest: two clients wanting the same drv build it once4720client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4721client # [ 67.027353] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4722client # Running as unit: dedup1.service; invocation ID: 99bb673007594c67975a9b5d2cef2bd54723client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4724client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4725worker2 # [ 67.200356] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457244726client # [ 67.227428] nix[935]: don't know how to build these paths:4727client # [ 67.228885] nix[935]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4728worker1 # [ 67.213358] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04729worker1 # [ 67.219481] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14730worker1 # [ 67.265764] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404731worker1 # [ 67.281716] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404732client # [ 67.319672] nix[935]: copying 1 paths to 'lb:50051'...4733worker1 # [ 67.296482] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430404734client # [ 67.322255] nix[935]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4735worker1 # [ 67.311309] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4736worker1 # [ 67.473609] nix-daemon[852]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)4737worker1 # [ 67.494323] nix-daemon[852]: reaped child process 1068, status = succeeded4738worker1 # [ 67.594502] nix-daemon[852]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4739worker1 # [ 67.610655] nix-daemon[852]: reaped child process 1073, status = succeeded4740lb # [ 67.573188] niks3-server[1033]: 2026/09/22 11:32:42 INFO Received uploads request method=POST path=/api/pending_closures4741worker1 # [ 67.627556] nix-grpc-daemon[867]: time=2026-09-22T11:32:42.222Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4742worker1 # [ 67.629179] nix-grpc-daemon[867]: time=2026-09-22T11:32:42.223Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4743lb # [ 67.600633] niks3-server[1033]: 2026/09/22 11:32:42 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4744lb # [ 67.617146] niks3-server[1033]: 2026/09/22 11:32:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4745worker1 # [ 67.668302] nix-grpc-daemon[867]: time=2026-09-22T11:32:42.263Z level=INFO msg="Uploading 1 narinfos"4746lb # [ 67.628308] niks3-server[1033]: 2026/09/22 11:32:42 INFO Signed narinfos id=15 count=14747lb # [ 67.634341] niks3-server[1033]: 2026/09/22 11:32:42 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4748lb # [ 67.645693] niks3-server[1033]: 2026/09/22 11:32:42 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4749lb # [ 67.795783] niks3-server[1033]: 2026/09/22 11:32:42 INFO Completed upload id=154750worker1 # [ 67.841643] nix-grpc-daemon[867]: time=2026-09-22T11:32:42.436Z level=INFO msg="Upload complete. (512ms)"4751lb # [ 67.803373] niks3-server[1033]: 2026/09/22 11:32:42 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4752worker1 # [ 67.852382] nix-grpc-daemon[768]: ts=2026-09-22T11:32:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=4324753worker1 # [ 67.871870] nix-grpc-daemon[768]: ts=2026-09-22T11:32:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404754worker1 # [ 67.884776] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4755worker1 # [ 67.889229] nix-daemon[852]: reaped child process 1063, status = succeeded4756client # [ 67.925072] nix[935]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4757worker1 # [ 67.906389] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4758client # [ 68.125675] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4759client # Running as unit: dedup2.service; invocation ID: 94343c2d804b43c795578cc119c16dc14760client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4761client # [ 68.341261] nix[951]: don't know how to build these paths:4762client # [ 68.343312] nix[951]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4763worker2 # [ 68.320757] nix-grpc-daemon[771]: ts=2026-09-22T11:32:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457244764worker1 # [ 68.329913] nix-grpc-daemon[768]: ts=2026-09-22T11:32:42Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04765client # [ 68.360848] nix[951]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4766worker1 # [ 68.337598] nix-grpc-daemon[768]: ts=2026-09-22T11:32:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404767lb # [ 68.328002] envoy[893]: conn peer=[2001:db8:1::1]:40630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1895 ms=18914768lb # [ 68.333120] envoy[893]: conn peer=[2001:db8:1::1]:40640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9558 tx=4408 ms=62964769lb # [ 68.338146] envoy[893]: conn peer=[2001:db8:1::1]:33778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2887 ms=2004770lb # [ 68.343348] envoy[893]: conn peer=[2001:db8:1::1]:33796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=864771lb # [ 68.348612] envoy[893]: conn peer=[2001:db8:1::1]:33788 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=1184772lb # [ 68.354252] envoy[893]: conn peer=[2001:db8:1::1]:33790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=1034773client: must succeed: sleep 24774client: (finished: must succeed: sleep 2, in 2.05 seconds)4775client: waiting for success: ! systemctl is-active dedup1 dedup24776worker1 # [ 71.771493] nix-daemon[852]: reaped child process 1077, status = succeeded4777worker1 # [ 71.911824] nix-daemon[852]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4778worker1 # [ 71.931442] nix-daemon[852]: reaped child process 1112, status = succeeded4779lb # [ 71.893406] niks3-server[1033]: 2026/09/22 11:32:46 INFO Received uploads request method=POST path=/api/pending_closures4780worker1 # [ 71.952146] nix-grpc-daemon[867]: time=2026-09-22T11:32:46.543Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4781worker1 # [ 71.953996] nix-grpc-daemon[867]: time=2026-09-22T11:32:46.543Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4782lb # [ 71.933383] niks3-server[1033]: 2026/09/22 11:32:46 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4783lb # [ 71.954245] niks3-server[1033]: 2026/09/22 11:32:46 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4784worker1 # [ 72.010934] nix-grpc-daemon[867]: time=2026-09-22T11:32:46.606Z level=INFO msg="Uploading 1 narinfos"4785lb # [ 71.966544] niks3-server[1033]: 2026/09/22 11:32:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4786lb # [ 71.976542] niks3-server[1033]: 2026/09/22 11:32:46 INFO Signed narinfos id=16 count=14787lb # [ 71.982218] niks3-server[1033]: 2026/09/22 11:32:46 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4788lb # [ 71.995293] niks3-server[1033]: 2026/09/22 11:32:46 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4789lb # [ 72.000352] niks3-server[1033]: 2026/09/22 11:32:46 INFO Completed upload id=164790worker1 # [ 72.045904] nix-grpc-daemon[867]: time=2026-09-22T11:32:46.641Z level=INFO msg="Upload complete. (321ms)"4791lb # [ 72.005257] niks3-server[1033]: 2026/09/22 11:32:46 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4792worker1 # [ 72.061158] nix-grpc-daemon[768]: ts=2026-09-22T11:32:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14793worker1 # [ 72.079360] nix-grpc-daemon[768]: ts=2026-09-22T11:32:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14794worker1 # [ 72.086920] nix-daemon[852]: reaped child process 1075, status = succeeded4795client # [ 72.114733] systemd[1]: dedup2.service: Deactivated successfully.4796client # [ 72.119724] systemd[1]: dedup2.service: Consumed 194ms CPU time over 3.991s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4797client # [ 72.159683] systemd[1]: dedup1.service: Deactivated successfully.4798client # [ 72.163232] systemd[1]: dedup1.service: Consumed 175ms CPU time over 5.135s wall clock time, 41.5M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic.4799client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4800client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4801client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4802worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4803worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4804worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4805worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4806worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4807worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4808worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4809worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4810(finished: subtest: two clients wanting the same drv build it once, in 6.13 seconds)4811subtest: --timeout and --max-silent-time reach the builder4812worker2 # [ 73.300872] nix-grpc-daemon[771]: ts=2026-09-22T11:32:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457244813worker1 # [ 73.308562] nix-grpc-daemon[768]: ts=2026-09-22T11:32:47Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04814worker2 # [ 73.315787] nix-grpc-daemon[771]: ts=2026-09-22T11:32:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244815worker1 # [ 73.308983] nix-grpc-daemon[768]: ts=2026-09-22T11:32:47Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14816worker2 # [ 73.330954] nix-grpc-daemon[771]: ts=2026-09-22T11:32:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457244817worker2 # [ 73.349452] nix-grpc-daemon[771]: ts=2026-09-22T11:32:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457244818worker2 # [ 73.365261] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4819worker2 # [ 73.528520] nix-daemon[842]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4820worker2 # [ 73.547331] nix-daemon[842]: reaped child process 1009, status = succeeded4821worker2 # [ 73.644750] nix-daemon[842]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4822worker2 # [ 73.660245] nix-daemon[842]: reaped child process 1014, status = succeeded4823lb # [ 73.618280] niks3-server[1033]: 2026/09/22 11:32:48 INFO Received uploads request method=POST path=/api/pending_closures4824worker2 # [ 73.680104] nix-grpc-daemon[867]: time=2026-09-22T11:32:48.266Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4825worker2 # [ 73.681569] nix-grpc-daemon[867]: time=2026-09-22T11:32:48.266Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4826lb # [ 73.650043] niks3-server[1033]: 2026/09/22 11:32:48 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4827worker2 # [ 73.714430] nix-grpc-daemon[867]: time=2026-09-22T11:32:48.301Z level=INFO msg="Uploading 1 narinfos"4828lb # [ 73.666017] niks3-server[1033]: 2026/09/22 11:32:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4829lb # [ 73.676644] niks3-server[1033]: 2026/09/22 11:32:48 INFO Signed narinfos id=17 count=14830lb # [ 73.684520] niks3-server[1033]: 2026/09/22 11:32:48 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4831lb # [ 73.695370] niks3-server[1033]: 2026/09/22 11:32:48 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4832worker2 # [ 73.749553] nix-grpc-daemon[867]: time=2026-09-22T11:32:48.336Z level=INFO msg="Upload complete. (368ms)"4833lb # [ 73.704275] niks3-server[1033]: 2026/09/22 11:32:48 INFO Completed upload id=174834lb # [ 73.707742] niks3-server[1033]: 2026/09/22 11:32:48 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4835worker2 # [ 73.760304] nix-grpc-daemon[771]: ts=2026-09-22T11:32:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=4244836worker2 # [ 73.775435] nix-grpc-daemon[771]: ts=2026-09-22T11:32:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244837worker2 # [ 73.782090] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4838worker2 # [ 73.786965] nix-daemon[842]: reaped child process 1004, status = succeeded4839worker2 # [ 73.797947] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4840lb # [ 78.327453] envoy[893]: conn peer=[2001:db8:1::1]:33810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=37594841lb # [ 78.327830] envoy[893]: conn peer=[2001:db8:1::1]:33802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1563 ms=49204842worker2 # [ 80.215279] nix-grpc-daemon[771]: ts=2026-09-22T11:32:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04843worker2 # [ 80.246647] nix-daemon[842]: reaped child process 1016, status = succeeded4844worker2 # [ 80.249415] nix-daemon[842]: reaped child process 1018, status = succeeded4845worker1 # [ 80.453709] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:430404846worker1 # [ 80.469608] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04847worker1 # [ 80.478890] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14848worker1 # [ 80.487626] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404849worker1 # [ 80.495395] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404850worker1 # [ 80.505838] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430404851worker1 # [ 80.521190] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4852worker1 # [ 80.680860] nix-daemon[852]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)4853worker1 # [ 80.706574] nix-daemon[852]: reaped child process 1132, status = succeeded4854worker1 # [ 80.798958] nix-daemon[852]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)4855worker1 # [ 80.817172] nix-daemon[852]: reaped child process 1137, status = succeeded4856lb # [ 80.771139] niks3-server[1033]: 2026/09/22 11:32:55 INFO Received uploads request method=POST path=/api/pending_closures4857worker1 # [ 80.826640] nix-grpc-daemon[867]: time=2026-09-22T11:32:55.421Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4858worker1 # [ 80.828118] nix-grpc-daemon[867]: time=2026-09-22T11:32:55.421Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4859lb # [ 80.808404] niks3-server[1033]: 2026/09/22 11:32:55 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4860worker1 # [ 80.865328] nix-grpc-daemon[867]: time=2026-09-22T11:32:55.460Z level=INFO msg="Uploading 1 narinfos"4861lb # [ 80.824245] niks3-server[1033]: 2026/09/22 11:32:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4862lb # [ 80.833248] niks3-server[1033]: 2026/09/22 11:32:55 INFO Signed narinfos id=18 count=14863worker1 # [ 80.891467] nix-grpc-daemon[867]: time=2026-09-22T11:32:55.487Z level=INFO msg="Upload complete. (355ms)"4864lb # [ 80.839588] niks3-server[1033]: 2026/09/22 11:32:55 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4865lb # [ 80.849801] niks3-server[1033]: 2026/09/22 11:32:55 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4866lb # [ 80.855324] niks3-server[1033]: 2026/09/22 11:32:55 INFO Completed upload id=184867worker1 # [ 80.900307] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=4244868lb # [ 80.859310] niks3-server[1033]: 2026/09/22 11:32:55 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4869worker1 # [ 80.912006] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404870worker1 # [ 80.920496] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4871worker1 # [ 80.923399] nix-daemon[852]: reaped child process 1127, status = succeeded4872worker1 # [ 80.936415] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4873worker1 # [ 87.553178] nix-grpc-daemon[768]: ts=2026-09-22T11:33:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04874worker1 # [ 87.593863] nix-daemon[852]: reaped child process 1139, status = succeeded4875worker1 # [ 87.597076] nix-daemon[852]: reaped child process 1141, status = succeeded4876(finished: subtest: --timeout and --max-silent-time reach the builder, in 14.70 seconds)4877subtest: the first client leaving does not fail the second4878client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4879client # [ 87.871556] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4880client # Running as unit: share1.service; invocation ID: 08686b9440c44176b31e9b420d663e9b4881client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.10 seconds)4882client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4883worker1 # [ 88.081026] nix-grpc-daemon[768]: ts=2026-09-22T11:33:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:430404884client # [ 88.111066] nix[1023]: don't know how to build these paths:4885client # [ 88.112187] nix[1023]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4886worker1 # [ 88.100149] nix-grpc-daemon[768]: ts=2026-09-22T11:33:02Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04887worker1 # [ 88.106489] nix-grpc-daemon[768]: ts=2026-09-22T11:33:02Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14888worker2 # [ 88.115835] nix-grpc-daemon[771]: ts=2026-09-22T11:33:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244889worker2 # [ 88.137823] nix-grpc-daemon[771]: ts=2026-09-22T11:33:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457244890client # [ 88.170823] nix[1023]: copying 1 paths to 'lb:50051'...4891client # [ 88.175038] nix[1023]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4892worker2 # [ 88.157298] nix-grpc-daemon[771]: ts=2026-09-22T11:33:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457244893worker2 # [ 88.177703] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4894worker2 # [ 88.345360] nix-daemon[842]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4895worker2 # [ 88.369281] nix-daemon[842]: reaped child process 1044, status = succeeded4896lb # [ 88.328364] envoy[893]: conn peer=[2001:db8:1::1]:49556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=978 ms=69414897worker2 # [ 88.476115] nix-daemon[842]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4898worker2 # [ 88.496269] nix-daemon[842]: reaped child process 1049, status = succeeded4899lb # [ 88.333414] envoy[893]: conn peer=[2001:db8:1::1]:49562 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=976 ms=71254900lb # [ 88.451186] niks3-server[1033]: 2026/09/22 11:33:03 INFO Received uploads request method=POST path=/api/pending_closures4901worker2 # [ 88.516155] nix-grpc-daemon[867]: time=2026-09-22T11:33:03.102Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4902worker2 # [ 88.517728] nix-grpc-daemon[867]: time=2026-09-22T11:33:03.102Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4903lb # [ 88.486247] niks3-server[1033]: 2026/09/22 11:33:03 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4904lb # [ 88.507896] niks3-server[1033]: 2026/09/22 11:33:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4905worker2 # [ 88.562252] nix-grpc-daemon[867]: time=2026-09-22T11:33:03.149Z level=INFO msg="Uploading 1 narinfos"4906lb # [ 88.516253] niks3-server[1033]: 2026/09/22 11:33:03 INFO Signed narinfos id=19 count=14907lb # [ 88.523990] niks3-server[1033]: 2026/09/22 11:33:03 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4908lb # [ 88.536164] niks3-server[1033]: 2026/09/22 11:33:03 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4909worker2 # [ 88.591924] nix-grpc-daemon[867]: time=2026-09-22T11:33:03.178Z level=INFO msg="Upload complete. (394ms)"4910lb # [ 88.544535] niks3-server[1033]: 2026/09/22 11:33:03 INFO Completed upload id=194911lb # [ 88.547774] niks3-server[1033]: 2026/09/22 11:33:03 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4912worker2 # [ 88.604445] nix-grpc-daemon[771]: ts=2026-09-22T11:33:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=4324913worker2 # [ 88.619563] nix-grpc-daemon[771]: ts=2026-09-22T11:33:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244914worker2 # [ 88.626846] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4915worker2 # [ 88.631684] nix-daemon[842]: reaped child process 1039, status = succeeded4916worker2 # [ 88.643862] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4917client # [ 88.661529] nix[1023]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4918client # [ 88.990682] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4919client # Running as unit: share2.service; invocation ID: 374f80440e844b26860911c9daf1753e4920client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds)4921worker2 # [ 89.246319] nix-grpc-daemon[771]: ts=2026-09-22T11:33:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457244922client # [ 89.295237] nix[1039]: this derivation will be built:4923worker1 # [ 89.279068] nix-grpc-daemon[768]: ts=2026-09-22T11:33:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:430404924worker1 # [ 89.304747] nix-grpc-daemon[768]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' from 'http://lb:5751'...4925client # [ 89.359652] nix[1039]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4926worker1 # [ 89.357725] nix-grpc-daemon[768]: ts=2026-09-22T11:33:03Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04927worker2 # [ 89.369250] nix-grpc-daemon[771]: ts=2026-09-22T11:33:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457244928client # [ 89.397027] nix[1039]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4929client: must succeed: sleep 2; systemctl kill -s INT share14930client # [ 92.385735] systemd[1]: share1.service: Sent signal SIGINT to main process 1023 (nix) on client request.4931client # [ 92.396518] nix[1023]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4932client # [ 92.409563] nix[1023]: error: interrupted by the user4933client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.10 seconds)4934client: waiting for success: ! systemctl is-active share14935client # [ 92.454329] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4936client # [ 92.458499] systemd[1]: share1.service: Failed with result 'exit-code'.4937client # [ 92.460940] systemd[1]: share1.service: Consumed 199ms CPU time over 4.586s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8.2K outgoing IP traffic.4938client: (finished: waiting for success: ! systemctl is-active share1, in 0.06 seconds)4939client: waiting for success: ! systemctl is-active share24940worker2 # [ 92.925043] nix-daemon[842]: reaped child process 1053, status = succeeded4941worker2 # [ 93.038184] nix-daemon[842]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)4942worker2 # [ 93.056961] nix-daemon[842]: reaped child process 1090, status = succeeded4943lb # [ 93.013926] niks3-server[1033]: 2026/09/22 11:33:07 INFO Received uploads request method=POST path=/api/pending_closures4944worker2 # [ 93.076443] nix-grpc-daemon[867]: time=2026-09-22T11:33:07.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4945worker2 # [ 93.078246] nix-grpc-daemon[867]: time=2026-09-22T11:33:07.661Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4946lb # [ 93.063482] niks3-server[1033]: 2026/09/22 11:33:07 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4947lb # [ 93.077700] niks3-server[1033]: 2026/09/22 11:33:07 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4948worker2 # [ 93.141012] nix-grpc-daemon[867]: time=2026-09-22T11:33:07.726Z level=INFO msg="Uploading 1 narinfos"4949lb # [ 93.093348] niks3-server[1033]: 2026/09/22 11:33:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4950lb # [ 93.102413] niks3-server[1033]: 2026/09/22 11:33:07 INFO Signed narinfos id=20 count=14951lb # [ 93.109635] niks3-server[1033]: 2026/09/22 11:33:07 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4952lb # [ 93.119833] niks3-server[1033]: 2026/09/22 11:33:07 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4953lb # [ 93.126843] niks3-server[1033]: 2026/09/22 11:33:07 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4954lb # [ 93.134597] niks3-server[1033]: 2026/09/22 11:33:07 INFO Completed upload id=204955worker2 # [ 93.183108] nix-grpc-daemon[867]: time=2026-09-22T11:33:07.770Z level=INFO msg="Upload complete. (317ms)"4956worker2 # [ 93.197121] nix-grpc-daemon[771]: ts=2026-09-22T11:33:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14957worker2 # [ 93.212446] nix-grpc-daemon[771]: ts=2026-09-22T11:33:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=3 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14958worker2 # [ 93.221535] nix-daemon[842]: reaped child process 1051, status = succeeded4959client # [ 93.289131] systemd[1]: share2.service: Deactivated successfully.4960client # [ 93.294230] systemd[1]: share2.service: Consumed 258ms CPU time over 4.300s wall clock time, 31.3M memory peak, 6.6K incoming IP traffic, 5.9K outgoing IP traffic.4961client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4962client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4963client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds)4964(finished: subtest: the first client leaving does not fail the second, in 6.10 seconds)4965subtest: build hook: nix-daemon with builders = grpc://lb4966client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4967client # [ 94.039373] systemd[1]: Started Nix Daemon.4968client # [ 94.157899] nix-daemon[1086]: accepted connection from pid 1083, user root (trusted)4969worker2 # [ 94.286511] nix-grpc-daemon[771]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:457244970worker1 # [ 94.286760] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404971worker1 # [ 94.305513] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04972worker1 # [ 94.312777] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14973worker1 # [ 94.321886] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404974worker1 # [ 94.333223] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430404975worker1 # [ 94.348786] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430404976worker1 # [ 94.365002] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4977worker1 # [ 94.521115] nix-daemon[852]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)4978worker1 # [ 94.541626] nix-daemon[852]: reaped child process 1190, status = succeeded4979worker1 # [ 94.640989] nix-daemon[852]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)4980worker1 # [ 94.659457] nix-daemon[852]: reaped child process 1195, status = succeeded4981lb # [ 94.619105] niks3-server[1033]: 2026/09/22 11:33:09 INFO Received uploads request method=POST path=/api/pending_closures4982worker1 # [ 94.673941] nix-grpc-daemon[867]: time=2026-09-22T11:33:09.268Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4983worker1 # [ 94.675441] nix-grpc-daemon[867]: time=2026-09-22T11:33:09.268Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4984lb # [ 94.648695] niks3-server[1033]: 2026/09/22 11:33:09 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4985lb # [ 94.661834] niks3-server[1033]: 2026/09/22 11:33:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4986worker1 # [ 94.711475] nix-grpc-daemon[867]: time=2026-09-22T11:33:09.306Z level=INFO msg="Uploading 1 narinfos"4987lb # [ 94.669679] niks3-server[1033]: 2026/09/22 11:33:09 INFO Signed narinfos id=21 count=14988lb # [ 94.675838] niks3-server[1033]: 2026/09/22 11:33:09 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4989lb # [ 94.684602] niks3-server[1033]: 2026/09/22 11:33:09 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4990worker1 # [ 94.733826] nix-grpc-daemon[867]: time=2026-09-22T11:33:09.329Z level=INFO msg="Upload complete. (352ms)"4991lb # [ 94.691375] niks3-server[1033]: 2026/09/22 11:33:09 INFO Completed upload id=214992lb # [ 94.696097] niks3-server[1033]: 2026/09/22 11:33:09 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4993worker1 # [ 94.744291] nix-grpc-daemon[768]: ts=2026-09-22T11:33:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=5124994worker1 # [ 94.755092] nix-grpc-daemon[768]: ts=2026-09-22T11:33:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430404995worker1 # [ 94.760798] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4996worker1 # [ 94.765536] nix-daemon[852]: reaped child process 1185, status = succeeded4997worker1 # [ 94.775268] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4998worker1 # [ 95.811968] nix-daemon[852]: reaped child process 1199, status = succeeded4999worker1 # [ 95.903032] nix-daemon[852]: accepted connection from pid 1215, user nix-grpc-daemon (trusted)5000worker1 # [ 95.923159] nix-daemon[852]: reaped child process 1218, status = succeeded5001lb # [ 95.881497] niks3-server[1033]: 2026/09/22 11:33:10 INFO Received uploads request method=POST path=/api/pending_closures5002worker1 # [ 95.936372] nix-grpc-daemon[867]: time=2026-09-22T11:33:10.530Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5003worker1 # [ 95.937807] nix-grpc-daemon[867]: time=2026-09-22T11:33:10.530Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"5004lb # [ 95.914325] niks3-server[1033]: 2026/09/22 11:33:10 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv5005lb # [ 95.934114] niks3-server[1033]: 2026/09/22 11:33:10 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5006worker1 # [ 95.989580] nix-grpc-daemon[867]: time=2026-09-22T11:33:10.584Z level=INFO msg="Uploading 1 narinfos"5007lb # [ 95.945423] niks3-server[1033]: 2026/09/22 11:33:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5008lb # [ 95.953633] niks3-server[1033]: 2026/09/22 11:33:10 INFO Signed narinfos id=22 count=15009lb # [ 95.958335] niks3-server[1033]: 2026/09/22 11:33:10 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls5010worker1 # [ 96.010414] nix-grpc-daemon[867]: time=2026-09-22T11:33:10.606Z level=INFO msg="Upload complete. (224ms)"5011lb # [ 95.967473] niks3-server[1033]: 2026/09/22 11:33:10 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5012lb # [ 95.974176] niks3-server[1033]: 2026/09/22 11:33:10 INFO Completed upload id=225013lb # [ 95.976398] niks3-server[1033]: 2026/09/22 11:33:10 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo5014worker1 # [ 96.021051] nix-grpc-daemon[768]: ts=2026-09-22T11:33:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=15015worker1 # [ 96.037999] nix-daemon[852]: reaped child process 1197, status = succeeded5016worker1 # [ 96.065340] nix-grpc-daemon[768]: ts=2026-09-22T11:33:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:430405017worker1 # [ 96.100458] nix-grpc-daemon[768]: ts=2026-09-22T11:33:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:430405018worker1 # [ 96.107345] nix-grpc-daemon[768]: ts=2026-09-22T11:33:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_out=1285019worker2 # [ 96.399880] nix-grpc-daemon[771]: ts=2026-09-22T11:33:10Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:457245020worker2 # [ 96.407789] nix-grpc-daemon[771]: ts=2026-09-22T11:33:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457245021worker1 # [ 96.425223] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=05022worker1 # [ 96.434444] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=15023worker1 # [ 96.443051] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430405024worker1 # [ 96.450568] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430405025worker1 # [ 96.461237] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.056Z level=INFO msg="All 1 paths already cached"5026worker1 # [ 96.470986] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430405027worker1 # [ 96.485025] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5028worker1 # [ 96.647032] nix-daemon[852]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5029worker1 # [ 96.667447] nix-daemon[852]: reaped child process 1227, status = succeeded5030worker1 # [ 96.766831] nix-daemon[852]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5031worker1 # [ 96.784798] nix-daemon[852]: reaped child process 1232, status = succeeded5032lb # [ 96.738836] niks3-server[1033]: 2026/09/22 11:33:11 INFO Received uploads request method=POST path=/api/pending_closures5033worker1 # [ 96.794488] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.389Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5034worker1 # [ 96.796230] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.389Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"5035lb # [ 96.771960] niks3-server[1033]: 2026/09/22 11:33:11 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst5036worker1 # [ 96.833339] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.428Z level=INFO msg="Uploading 1 narinfos"5037lb # [ 96.789805] niks3-server[1033]: 2026/09/22 11:33:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5038lb # [ 96.799565] niks3-server[1033]: 2026/09/22 11:33:11 INFO Signed narinfos id=23 count=15039lb # [ 96.806801] niks3-server[1033]: 2026/09/22 11:33:11 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls5040worker1 # [ 96.864084] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.459Z level=INFO msg="Upload complete. (362ms)"5041lb # [ 96.821624] niks3-server[1033]: 2026/09/22 11:33:11 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5042worker1 # [ 96.872451] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=6245043lb # [ 96.830325] niks3-server[1033]: 2026/09/22 11:33:11 INFO Completed upload id=235044lb # [ 96.833360] niks3-server[1033]: 2026/09/22 11:33:11 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo5045worker1 # [ 96.884994] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430405046worker1 # [ 96.891555] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5047worker1 # [ 96.897286] nix-daemon[852]: reaped child process 1222, status = succeeded5048worker1 # [ 96.909153] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5049worker1 # [ 97.970723] nix-daemon[852]: reaped child process 1236, status = succeeded5050worker1 # [ 98.098205] nix-daemon[852]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5051worker1 # [ 98.116222] nix-daemon[852]: reaped child process 1255, status = succeeded5052lb # [ 98.078701] niks3-server[1033]: 2026/09/22 11:33:12 INFO Received uploads request method=POST path=/api/pending_closures5053worker1 # [ 98.136456] nix-grpc-daemon[867]: time=2026-09-22T11:33:12.729Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5054worker1 # [ 98.138252] nix-grpc-daemon[867]: time=2026-09-22T11:33:12.729Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"5055lb # [ 98.119544] niks3-server[1033]: 2026/09/22 11:33:12 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst5056lb # [ 98.140786] niks3-server[1033]: 2026/09/22 11:33:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5057worker1 # [ 98.191107] nix-grpc-daemon[867]: time=2026-09-22T11:33:12.786Z level=INFO msg="Uploading 1 narinfos"5058lb # [ 98.152714] niks3-server[1033]: 2026/09/22 11:33:12 INFO Signed narinfos id=24 count=15059lb # [ 98.159090] niks3-server[1033]: 2026/09/22 11:33:12 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv5060lb # [ 98.168260] niks3-server[1033]: 2026/09/22 11:33:12 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls5061lb # [ 98.176642] niks3-server[1033]: 2026/09/22 11:33:12 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5062lb # [ 98.183553] niks3-server[1033]: 2026/09/22 11:33:12 INFO Completed upload id=245063worker1 # [ 98.228241] nix-grpc-daemon[867]: time=2026-09-22T11:33:12.823Z level=INFO msg="Upload complete. (308ms)"5064lb # [ 98.187190] niks3-server[1033]: 2026/09/22 11:33:12 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo5065worker1 # [ 98.244755] nix-grpc-daemon[768]: ts=2026-09-22T11:33:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=15066worker2 # [ 98.251975] nix-grpc-daemon[771]: ts=2026-09-22T11:33:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:457245067worker1 # [ 98.260865] nix-daemon[852]: reaped child process 1234, status = succeeded5068worker2 # [ 98.262565] nix-grpc-daemon[771]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...5069worker2 # [ 98.278475] nix-grpc-daemon[771]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...5070worker2 # [ 98.317030] nix-grpc-daemon[771]: ts=2026-09-22T11:33:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:457245071worker2 # [ 98.323866] nix-grpc-daemon[771]: ts=2026-09-22T11:33:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=1765072lb # [ 98.327668] envoy[893]: conn peer=[2001:db8:1::1]:41478 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2977 tx=814 ms=43175073lb # [ 98.332815] envoy[893]: conn peer=[2001:db8:1::1]:41482 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1592 tx=1599 ms=40125074lb # [ 98.337631] envoy[893]: conn peer=[2001:db8:1::1]:35630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3400 tx=1965 ms=18875075lb # [ 98.342289] envoy[893]: conn peer=[2001:db8:1::1]:35638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=765076lb # [ 98.346918] envoy[893]: conn peer=[2001:db8:1::1]:35640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3886 tx=2180 ms=19835077lb # [ 98.351713] envoy[893]: conn peer=[2001:db8:1::1]:35652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=695078worker2 # [ 98.599271] nix-grpc-daemon[771]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:457245079worker2 # [ 98.606859] nix-grpc-daemon[771]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457245080worker2 # [ 98.614775] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.199Z level=INFO msg="All 2 paths already cached"5081worker1 # [ 98.622455] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=05082worker1 # [ 98.628174] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=15083worker1 # [ 98.632653] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430405084worker1 # [ 98.637009] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430405085worker1 # [ 98.643878] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.239Z level=INFO msg="All 2 paths already cached"5086worker1 # [ 98.653858] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430405087worker1 # [ 98.668613] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5088worker1 # [ 98.819874] nix-daemon[852]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5089worker1 # [ 98.835999] nix-daemon[852]: reaped child process 1264, status = succeeded5090worker1 # [ 98.938691] nix-daemon[852]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5091worker1 # [ 98.953760] nix-daemon[852]: reaped child process 1269, status = succeeded5092lb # [ 98.914863] niks3-server[1033]: 2026/09/22 11:33:13 INFO Received uploads request method=POST path=/api/pending_closures5093worker1 # [ 98.969230] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.564Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5094worker1 # [ 98.970713] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.564Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"5095lb # [ 98.944278] niks3-server[1033]: 2026/09/22 11:33:13 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst5096worker1 # [ 99.000951] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.596Z level=INFO msg="Uploading 1 narinfos"5097lb # [ 98.957783] niks3-server[1033]: 2026/09/22 11:33:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5098lb # [ 98.969801] niks3-server[1033]: 2026/09/22 11:33:13 INFO Signed narinfos id=25 count=15099lb # [ 98.974818] niks3-server[1033]: 2026/09/22 11:33:13 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls5100worker1 # [ 99.030432] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.626Z level=INFO msg="Upload complete. (347ms)"5101lb # [ 98.985818] niks3-server[1033]: 2026/09/22 11:33:13 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5102worker1 # [ 99.036244] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=7525103lb # [ 98.995631] niks3-server[1033]: 2026/09/22 11:33:13 INFO Completed upload id=255104lb # [ 98.998305] niks3-server[1033]: 2026/09/22 11:33:13 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo5105worker1 # [ 99.045578] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430405106worker1 # [ 99.055967] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5107worker1 # [ 99.061419] nix-daemon[852]: reaped child process 1259, status = succeeded5108worker1 # [ 99.072182] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5109worker1 # [ 100.323922] nix-daemon[852]: reaped child process 1273, status = succeeded5110worker1 # [ 100.448228] nix-daemon[852]: accepted connection from pid 1289, user nix-grpc-daemon (trusted)5111worker1 # [ 100.465450] nix-daemon[852]: reaped child process 1292, status = succeeded5112lb # [ 100.430459] niks3-server[1033]: 2026/09/22 11:33:15 INFO Received uploads request method=POST path=/api/pending_closures5113worker1 # [ 100.484079] nix-grpc-daemon[867]: time=2026-09-22T11:33:15.078Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5114worker1 # [ 100.485767] nix-grpc-daemon[867]: time=2026-09-22T11:33:15.078Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"5115lb # [ 100.464561] niks3-server[1033]: 2026/09/22 11:33:15 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv5116worker1 # [ 100.525127] nix-grpc-daemon[867]: time=2026-09-22T11:33:15.120Z level=INFO msg="Uploading 1 narinfos"5117lb # [ 100.480788] niks3-server[1033]: 2026/09/22 11:33:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5118lb # [ 100.488455] niks3-server[1033]: 2026/09/22 11:33:15 INFO Signed narinfos id=26 count=15119lb # [ 100.495221] niks3-server[1033]: 2026/09/22 11:33:15 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst5120lb # [ 100.509336] niks3-server[1033]: 2026/09/22 11:33:15 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls5121worker1 # [ 100.558242] nix-grpc-daemon[867]: time=2026-09-22T11:33:15.153Z level=INFO msg="Upload complete. (274ms)"5122lb # [ 100.516115] niks3-server[1033]: 2026/09/22 11:33:15 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5123lb # [ 100.520083] niks3-server[1033]: 2026/09/22 11:33:15 INFO Completed upload id=265124lb # [ 100.525556] niks3-server[1033]: 2026/09/22 11:33:15 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo5125worker1 # [ 100.571641] nix-grpc-daemon[768]: ts=2026-09-22T11:33:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=15126worker1 # [ 100.592371] nix-grpc-daemon[768]: ts=2026-09-22T11:33:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:430405127worker1 # [ 100.598123] nix-daemon[852]: reaped child process 1271, status = succeeded5128worker2 # [ 100.612646] nix-grpc-daemon[771]: ts=2026-09-22T11:33:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:457245129worker2 # [ 100.626318] nix-grpc-daemon[771]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...5130worker2 # [ 100.642047] nix-grpc-daemon[771]: ts=2026-09-22T11:33:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=2325131client # [ 100.787485] nix-daemon[1086]: reaped child process 1088, status = succeeded5132client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.91 seconds)5133client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top5134client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)5135client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5136client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)5137client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5138client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5139(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.03 seconds)5140subtest: an input only one worker has still reaches the builder5141client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only5142client # this derivation will be built:5143client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv5144client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...5145client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)5146client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure5147client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)5148worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5149worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)5150worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only5151worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)5152client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&25153client # [ 101.744566] nix-daemon[1086]: accepted connection from pid 1169, user root (trusted)5154client # this derivation will be built:5155client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5156client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5157worker2 # [ 101.874923] nix-grpc-daemon[771]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:457245158worker1 # [ 101.875649] nix-grpc-daemon[768]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430405159worker1 # [ 102.045571] nix-daemon[852]: accepted connection from pid 1299, user nix-grpc-daemon (trusted)5160worker1 # [ 102.064516] nix-daemon[852]: reaped child process 1302, status = succeeded5161lb # [ 102.025114] niks3-server[1033]: 2026/09/22 11:33:16 INFO Received uploads request method=POST path=/api/pending_closures5162worker1 # [ 102.080296] nix-grpc-daemon[867]: time=2026-09-22T11:33:16.675Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5163worker1 # [ 102.082055] nix-grpc-daemon[867]: time=2026-09-22T11:33:16.675Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"5164lb # [ 102.052824] niks3-server[1033]: 2026/09/22 11:33:16 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5165worker1 # [ 102.111241] nix-grpc-daemon[867]: time=2026-09-22T11:33:16.706Z level=INFO msg="Uploading 1 narinfos"5166lb # [ 102.067816] niks3-server[1033]: 2026/09/22 11:33:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5167lb # [ 102.078196] niks3-server[1033]: 2026/09/22 11:33:16 INFO Signed narinfos id=27 count=15168lb # [ 102.084307] niks3-server[1033]: 2026/09/22 11:33:16 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls5169lb # [ 102.094000] niks3-server[1033]: 2026/09/22 11:33:16 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5170worker1 # [ 102.143305] nix-grpc-daemon[867]: time=2026-09-22T11:33:16.738Z level=INFO msg="Upload complete. (261ms)"5171lb # [ 102.100385] niks3-server[1033]: 2026/09/22 11:33:16 INFO Completed upload id=275172lb # [ 102.103125] niks3-server[1033]: 2026/09/22 11:33:16 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo5173worker1 # [ 102.169998] nix-grpc-daemon[768]: ts=2026-09-22T11:33:16Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=05174worker1 # [ 102.176524] nix-grpc-daemon[768]: ts=2026-09-22T11:33:16Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15175worker2 # [ 102.221582] nix-grpc-daemon[771]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457245176worker2 # [ 102.236691] nix-grpc-daemon[771]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457245177client # copying 2 paths to 'lb:50051'...5178client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...5179worker2 # [ 102.254769] nix-grpc-daemon[771]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457245180client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...5181worker2 # [ 102.269809] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5182worker2 # [ 102.438792] nix-daemon[842]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)5183worker2 # [ 102.456609] nix-daemon[842]: reaped child process 1103, status = succeeded5184worker2 # [ 102.555783] nix-daemon[842]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5185worker2 # [ 102.569838] nix-daemon[842]: reaped child process 1108, status = succeeded5186lb # [ 102.528532] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received uploads request method=POST path=/api/pending_closures5187lb # [ 102.537727] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received uploads request method=POST path=/api/pending_closures5188worker2 # [ 102.595915] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.182Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5189worker2 # [ 102.597554] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.182Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"5190worker2 # [ 102.599253] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.182Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"5191lb # [ 102.577854] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst5192lb # [ 102.594376] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5193worker2 # [ 102.652134] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.238Z level=INFO msg="Uploading 3 narinfos"5194lb # [ 102.607733] niks3-server[1033]: 2026/09/22 11:33:17 INFO Signed narinfos id=29 count=25195lb # [ 102.613554] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5196lb # [ 102.619178] niks3-server[1033]: 2026/09/22 11:33:17 INFO Signed narinfos id=28 count=15197lb # [ 102.624362] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5198lb # [ 102.627013] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst5199lb # [ 102.637975] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls5200lb # [ 102.644736] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5201lb # [ 102.651631] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5202lb # [ 102.658358] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5203lb # [ 102.662708] niks3-server[1033]: 2026/09/22 11:33:17 INFO Completed upload id=285204lb # [ 102.666233] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5205worker2 # [ 102.714795] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.301Z level=INFO msg="Upload complete. (425ms)"5206lb # [ 102.668868] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5207lb # [ 102.671587] niks3-server[1033]: 2026/09/22 11:33:17 INFO Completed upload id=295208worker2 # [ 102.724517] nix-grpc-daemon[771]: ts=2026-09-22T11:33:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=2 nar_bytes_in=9845209worker2 # [ 102.740912] nix-grpc-daemon[771]: ts=2026-09-22T11:33:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457245210worker2 # [ 102.755266] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5211worker2 # [ 102.761645] nix-daemon[842]: reaped child process 1098, status = succeeded5212worker2 # [ 102.778464] nix-grpc-daemon[771]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...5213client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5214worker2 # [ 102.794631] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5215client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5216client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5217worker2 # [ 104.041080] nix-daemon[842]: reaped child process 1113, status = succeeded5218worker2 # [ 104.139516] nix-daemon[842]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)5219worker2 # [ 104.156769] nix-daemon[842]: reaped child process 1132, status = succeeded5220lb # [ 104.114525] niks3-server[1033]: 2026/09/22 11:33:18 INFO Received uploads request method=POST path=/api/pending_closures5221worker2 # [ 104.174387] nix-grpc-daemon[867]: time=2026-09-22T11:33:18.759Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5222lb # [ 104.145254] niks3-server[1033]: 2026/09/22 11:33:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5223worker2 # [ 104.202691] nix-grpc-daemon[867]: time=2026-09-22T11:33:18.789Z level=INFO msg="Uploading 1 narinfos"5224lb # [ 104.160479] niks3-server[1033]: 2026/09/22 11:33:18 INFO Signed narinfos id=30 count=15225lb # [ 104.168812] niks3-server[1033]: 2026/09/22 11:33:18 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5226lb # [ 104.182028] niks3-server[1033]: 2026/09/22 11:33:18 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5227lb # [ 104.185615] niks3-server[1033]: 2026/09/22 11:33:18 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5228lb # [ 104.191041] niks3-server[1033]: 2026/09/22 11:33:18 INFO Completed upload id=305229worker2 # [ 104.239128] nix-grpc-daemon[867]: time=2026-09-22T11:33:18.826Z level=INFO msg="Upload complete. (228ms)"5230lb # [ 104.195238] niks3-server[1033]: 2026/09/22 11:33:18 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5231worker2 # [ 104.248961] nix-grpc-daemon[771]: ts=2026-09-22T11:33:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15232worker2 # [ 104.264798] nix-daemon[842]: reaped child process 1110, status = succeeded5233client # copying 1 paths...5234worker2 # [ 104.298824] nix-grpc-daemon[771]: ts=2026-09-22T11:33:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:457245235client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5236worker2 # [ 104.330193] nix-grpc-daemon[771]: ts=2026-09-22T11:33:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:457245237worker2 # [ 104.337022] nix-grpc-daemon[771]: ts=2026-09-22T11:33:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=1205238client # [ 104.478515] nix-daemon[1086]: reaped child process 1172, status = succeeded5239client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.93 seconds)5240client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25241client # [ 104.654316] nix-daemon[1086]: accepted connection from pid 1191, user root (trusted)5242client # this derivation will be built:5243client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5244worker2 # [ 104.780106] nix-grpc-daemon[771]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:457245245client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5246worker2 # [ 104.787650] nix-grpc-daemon[771]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457245247worker2 # [ 104.795031] nix-grpc-daemon[867]: time=2026-09-22T11:33:19.380Z level=INFO msg="All 1 paths already cached"5248worker1 # [ 104.807136] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05249worker1 # [ 104.807484] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15250worker1 # [ 104.807646] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430405251worker1 # [ 104.830398] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430405252client # copying 1 paths to 'lb:50051'...5253client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5254worker1 # [ 104.850668] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430405255worker1 # [ 104.865765] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5256worker1 # [ 104.886889] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)5257worker1 # [ 104.973773] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 2/5)5258worker1 # [ 105.166582] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 3/5)5259worker1 # [ 105.265401] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 393 ms (attempt 4/5)5260worker1 # [ 105.660863] nix-grpc-daemon[768]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5261worker1 # [ 105.669245] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5262worker1 # [ 105.676652] nix-grpc-daemon[768]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5263worker1 # [ 105.840233] nix-daemon[852]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5264worker1 # [ 105.860312] nix-daemon[852]: reaped child process 1318, status = succeeded5265worker1 # [ 105.961744] nix-daemon[852]: accepted connection from pid 1320, user nix-grpc-daemon (trusted)5266worker1 # [ 105.977766] nix-daemon[852]: reaped child process 1323, status = succeeded5267lb # [ 105.939408] niks3-server[1033]: 2026/09/22 11:33:20 INFO Received uploads request method=POST path=/api/pending_closures5268worker1 # [ 105.994188] nix-grpc-daemon[867]: time=2026-09-22T11:33:20.589Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5269worker1 # [ 105.995656] nix-grpc-daemon[867]: time=2026-09-22T11:33:20.589Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5270lb # [ 105.966486] niks3-server[1033]: 2026/09/22 11:33:20 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5271lb # [ 105.981946] niks3-server[1033]: 2026/09/22 11:33:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5272worker1 # [ 106.034350] nix-grpc-daemon[867]: time=2026-09-22T11:33:20.629Z level=INFO msg="Uploading 1 narinfos"5273lb # [ 105.995318] niks3-server[1033]: 2026/09/22 11:33:20 INFO Signed narinfos id=31 count=15274lb # [ 106.002582] niks3-server[1033]: 2026/09/22 11:33:20 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5275worker1 # [ 106.060831] nix-grpc-daemon[867]: time=2026-09-22T11:33:20.656Z level=INFO msg="Upload complete. (371ms)"5276lb # [ 106.016559] niks3-server[1033]: 2026/09/22 11:33:20 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5277worker1 # [ 106.068707] nix-grpc-daemon[768]: ts=2026-09-22T11:33:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 paths=1 nar_bytes_in=5685278lb # [ 106.026796] niks3-server[1033]: 2026/09/22 11:33:20 INFO Completed upload id=315279lb # [ 106.030515] niks3-server[1033]: 2026/09/22 11:33:20 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5280worker1 # [ 106.079946] nix-grpc-daemon[768]: ts=2026-09-22T11:33:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430405281worker1 # [ 106.090650] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5282worker1 # [ 106.096358] nix-daemon[852]: reaped child process 1308, status = succeeded5283client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5284worker1 # [ 106.107943] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5285client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5286client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5287worker1 # [ 107.569747] nix-daemon[852]: reaped child process 1327, status = succeeded5288worker1 # [ 107.701763] nix-daemon[852]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5289worker1 # [ 107.718104] nix-daemon[852]: reaped child process 1346, status = succeeded5290lb # [ 107.681839] niks3-server[1033]: 2026/09/22 11:33:22 INFO Received uploads request method=POST path=/api/pending_closures5291worker1 # [ 107.739297] nix-grpc-daemon[867]: time=2026-09-22T11:33:22.331Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5292worker1 # [ 107.771200] nix-grpc-daemon[867]: time=2026-09-22T11:33:22.366Z level=INFO msg="Uploading 1 narinfos"5293lb # [ 107.715010] niks3-server[1033]: 2026/09/22 11:33:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5294lb # [ 107.731501] niks3-server[1033]: 2026/09/22 11:33:22 INFO Signed narinfos id=32 count=15295lb # [ 107.738059] niks3-server[1033]: 2026/09/22 11:33:22 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5296lb # [ 107.751409] niks3-server[1033]: 2026/09/22 11:33:22 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5297worker1 # [ 107.805055] nix-grpc-daemon[867]: time=2026-09-22T11:33:22.400Z level=INFO msg="Upload complete. (287ms)"5298lb # [ 107.763145] niks3-server[1033]: 2026/09/22 11:33:22 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5299lb # [ 107.769226] niks3-server[1033]: 2026/09/22 11:33:22 INFO Completed upload id=325300lb # [ 107.772377] niks3-server[1033]: 2026/09/22 11:33:22 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5301worker1 # [ 107.817142] nix-grpc-daemon[768]: ts=2026-09-22T11:33:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15302worker1 # [ 107.837135] nix-daemon[852]: reaped child process 1325, status = succeeded5303client # copying 1 paths...5304worker2 # [ 107.867748] nix-grpc-daemon[771]: ts=2026-09-22T11:33:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:457245305worker2 # [ 107.880941] nix-grpc-daemon[771]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5306client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5307worker2 # [ 107.924278] nix-grpc-daemon[771]: ts=2026-09-22T11:33:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:457245308worker2 # [ 107.931409] nix-grpc-daemon[771]: ts=2026-09-22T11:33:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=1205309client # [ 108.077995] nix-daemon[1086]: reaped child process 1194, status = succeeded5310client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.60 seconds)5311client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25312client # [ 108.253133] nix-daemon[1086]: accepted connection from pid 1212, user root (trusted)5313client # this derivation will be built:5314client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5315lb # [ 108.329280] envoy[893]: conn peer=[2001:db8:1::1]:35662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=925316lb # [ 108.334532] envoy[893]: conn peer=[2001:db8:1::1]:35658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4337 tx=2389 ms=21025317lb # [ 108.340433] envoy[893]: conn peer=[2001:db8:1::1]:36600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3879 tx=2084 ms=25195318lb # [ 108.346525] envoy[893]: conn peer=[2001:db8:1::1]:36604 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=665319worker2 # [ 108.396403] nix-grpc-daemon[771]: ts=2026-09-22T11:33:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:457245320client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5321lb # [ 108.351988] envoy[893]: conn peer=[2001:db8:1::1]:36618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3752 tx=2125 ms=32095322worker1 # [ 108.403104] nix-grpc-daemon[768]: ts=2026-09-22T11:33:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430405323lb # [ 108.357845] envoy[893]: conn peer=[2001:db8:1::1]:36634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=695324worker1 # [ 108.411426] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.007Z level=INFO msg="All 1 paths already cached"5325worker1 # [ 108.428764] nix-grpc-daemon[768]: ts=2026-09-22T11:33:23Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05326worker1 # [ 108.435398] nix-grpc-daemon[768]: ts=2026-09-22T11:33:23Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15327worker2 # [ 108.442773] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457245328worker2 # [ 108.458298] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457245329client # copying 1 paths to 'lb:50051'...5330worker2 # [ 108.474952] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.059Z level=INFO msg="All 1 paths already cached"5331client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5332worker2 # [ 108.480702] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457245333worker2 # [ 108.495356] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5334worker2 # [ 108.660276] nix-daemon[842]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)5335worker2 # [ 108.679994] nix-daemon[842]: reaped child process 1141, status = succeeded5336worker2 # [ 108.776809] nix-daemon[842]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)5337worker2 # [ 108.791617] nix-daemon[842]: reaped child process 1146, status = succeeded5338lb # [ 108.750219] niks3-server[1033]: 2026/09/22 11:33:23 INFO Received uploads request method=POST path=/api/pending_closures5339worker2 # [ 108.809686] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.396Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5340worker2 # [ 108.811159] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.396Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5341lb # [ 108.779358] niks3-server[1033]: 2026/09/22 11:33:23 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5342worker2 # [ 108.843454] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.430Z level=INFO msg="Uploading 1 narinfos"5343lb # [ 108.795914] niks3-server[1033]: 2026/09/22 11:33:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5344lb # [ 108.807777] niks3-server[1033]: 2026/09/22 11:33:23 INFO Signed narinfos id=33 count=15345lb # [ 108.813345] niks3-server[1033]: 2026/09/22 11:33:23 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5346worker2 # [ 108.873276] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.460Z level=INFO msg="Upload complete. (360ms)"5347lb # [ 108.822702] niks3-server[1033]: 2026/09/22 11:33:23 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5348lb # [ 108.828110] niks3-server[1033]: 2026/09/22 11:33:23 INFO Completed upload id=335349lb # [ 108.832470] niks3-server[1033]: 2026/09/22 11:33:23 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5350worker2 # [ 108.881162] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=5685351worker2 # [ 108.895155] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457245352worker2 # [ 108.904268] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5353worker2 # [ 108.906889] nix-daemon[842]: reaped child process 1136, status = succeeded5354client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5355worker2 # [ 108.921649] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5356client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5357client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5358worker2 # [ 109.980468] nix-daemon[842]: reaped child process 1150, status = succeeded5359worker2 # [ 110.078028] nix-daemon[842]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)5360worker2 # [ 110.096622] nix-daemon[842]: reaped child process 1169, status = succeeded5361lb # [ 110.050487] niks3-server[1033]: 2026/09/22 11:33:24 INFO Received uploads request method=POST path=/api/pending_closures5362worker2 # [ 110.108343] nix-grpc-daemon[867]: time=2026-09-22T11:33:24.695Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5363lb # [ 110.083109] niks3-server[1033]: 2026/09/22 11:33:24 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5364worker2 # [ 110.143544] nix-grpc-daemon[867]: time=2026-09-22T11:33:24.730Z level=INFO msg="Uploading 1 narinfos"5365lb # [ 110.095158] niks3-server[1033]: 2026/09/22 11:33:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5366lb # [ 110.107137] niks3-server[1033]: 2026/09/22 11:33:24 INFO Signed narinfos id=34 count=15367lb # [ 110.115654] niks3-server[1033]: 2026/09/22 11:33:24 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5368worker2 # [ 110.170495] nix-grpc-daemon[867]: time=2026-09-22T11:33:24.757Z level=INFO msg="Upload complete. (236ms)"5369lb # [ 110.122760] niks3-server[1033]: 2026/09/22 11:33:24 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5370lb # [ 110.129663] niks3-server[1033]: 2026/09/22 11:33:24 INFO Completed upload id=345371lb # [ 110.131971] niks3-server[1033]: 2026/09/22 11:33:24 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5372worker2 # [ 110.180982] nix-grpc-daemon[771]: ts=2026-09-22T11:33:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15373worker2 # [ 110.196439] nix-daemon[842]: reaped child process 1148, status = succeeded5374client # copying 1 paths...5375worker1 # [ 110.223343] nix-grpc-daemon[768]: ts=2026-09-22T11:33:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:430405376worker1 # [ 110.236544] nix-grpc-daemon[768]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5377client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5378worker2 # [ 110.283021] nix-grpc-daemon[771]: ts=2026-09-22T11:33:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:457245379worker2 # [ 110.290267] nix-grpc-daemon[771]: ts=2026-09-22T11:33:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=1205380client # [ 110.391685] nix-daemon[1086]: reaped child process 1215, status = succeeded5381client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.31 seconds)5382(finished: subtest: an input only one worker has still reaches the builder, in 9.48 seconds)5383subtest: upload whose reference the worker lacks is completed from the cache5384client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5385client # this derivation will be built:5386client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5387client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5388client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5389client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5390client # this derivation will be built:5391client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5392client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5393client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)5394client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5395client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds)5396worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5397worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)5398worker2 # [ 111.359051] systemd[1]: Stopping Nix worker-protocol over gRPC...5399worker2 # [ 111.763161] nix-grpc-daemon[771]: ts=2026-09-22T11:33:26Z level=info event=signal_exit5400worker2 # [ 113.147823] nix-grpc-daemon[771]: ts=2026-09-22T11:33:27Z level=info event=scheduler_restarting5401worker1 # [ 113.144310] nix-grpc-daemon[768]: ts=2026-09-22T11:33:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:430345402worker1 # [ 113.144629] nix-grpc-daemon[768]: ts=2026-09-22T11:33:27Z level=info event=worker_gone addr=192.168.1.4:500515403worker2 # [ 113.180297] nix-grpc-daemon[771]: ts=2026-09-22T11:33:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5404worker2 # [ 113.201007] nix-daemon[842]: reaped child process 844, status = succeeded5405worker2 # [ 113.916641] nix-grpc-daemon[771]: E0000 00:00:1790076808.503431 816 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5406worker2 # [ 113.949625] nix-grpc-daemon[771]: E0000 00:00:1790076808.536530 816 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5407worker2 # [ 115.148971] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5408worker2 # [ 115.154135] systemd[1]: Stopped Nix worker-protocol over gRPC.5409worker2 # [ 115.160574] systemd[1]: nix-grpc-daemon.service: Consumed 3.387s CPU time over 1min 36.637s wall clock time, 58.6M memory peak, 88.9K incoming IP traffic, 233.4K outgoing IP traffic.5410worker2 # [ 115.176465] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5411worker2 # [ 115.183948] systemd[1]: Closed Nix worker-protocol over gRPC.5412worker2 # [ 115.186611] systemd[1]: nix-grpc-daemon.socket: Consumed 632us CPU time over 1min 44.772s wall clock time, 256K memory peak, 101.3K incoming IP traffic, 88.6K outgoing IP traffic.5413lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5414lb: must succeed: curl -sf localhost:9901/clusters5415lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5416lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5417client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25418worker1 # [ 115.499798] nix-grpc-daemon[768]: ts=2026-09-22T11:33:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:430405419client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5420worker1 # [ 115.506857] nix-grpc-daemon[768]: ts=2026-09-22T11:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:430405421client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.26 seconds)5422worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5423worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5424worker2 # [ 115.626121] systemd[1]: Listening on Nix worker-protocol over gRPC.5425worker1 # [ 115.695406] systemd[1]: Stopping Nix worker-protocol over gRPC...5426worker2 # [ 116.147897] systemd[1]: Starting scheduler-token.service...5427worker2 # [ 116.238547] systemd[1]: scheduler-token.service: Deactivated successfully.5428worker2 # [ 116.249977] systemd[1]: Finished scheduler-token.service.5429worker2 # [ 116.251856] systemd[1]: scheduler-token.service: Consumed 61ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5430worker2 # [ 116.258160] systemd[1]: Starting Nix worker-protocol over gRPC...5431worker2 # [ 116.383587] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5432worker2 # [ 116.385940] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5433worker2 # [ 116.389752] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15434worker2 # [ 116.396056] nix-grpc-daemon[1193]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5435worker2 # [ 116.397488] nix-grpc-daemon[1193]: E0000 00:00:1790076810.981250 1193 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5436worker2 # [ 116.440104] nix-grpc-daemon[1193]: E0000 00:00:1790076810.987462 1193 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5437worker2 # [ 116.442417] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5438worker2 # [ 116.448262] nix-grpc-daemon[1193]: E0000 00:00:1790076810.997876 1235 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5439worker2 # [ 116.451088] nix-grpc-daemon[1193]: E0000 00:00:1790076811.015954 1235 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5440worker2 # [ 116.453627] nix-grpc-daemon[1193]: E0000 00:00:1790076811.016081 1235 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5441worker2 # [ 116.455973] systemd[1]: Started Nix worker-protocol over gRPC.5442worker2 # [ 116.456839] nix-grpc-daemon[1193]: E0000 00:00:1790076811.034582 1235 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5443worker2 # [ 116.464247] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:31Z level=info event=scheduler_connected addr=lb:500515444worker1 # [ 116.462597] nix-grpc-daemon[768]: ts=2026-09-22T11:33:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:430345445worker1 # [ 116.468510] nix-grpc-daemon[768]: ts=2026-09-22T11:33:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05446worker1 # [ 116.671743] nix-grpc-daemon[768]: ts=2026-09-22T11:33:31Z level=info event=signal_exit5447lb # [ 117.090031] niks3-server[1033]: 2026/09/22 11:33:31 INFO lead: released remote=192.168.1.3:445025448worker1 # [ 118.134502] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=scheduler_restarting5449worker2 # [ 118.141011] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:32Z level=info event=scheduler_restarting addr=lb:500515450worker1 # [ 118.146324] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:430345451worker1 # [ 118.151623] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=worker_gone addr=192.168.1.4:500515452worker1 # [ 118.156578] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=scheduler_restarting addr=lb:500515453worker1 # [ 118.162011] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:430345454worker1 # [ 118.167123] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=worker_gone addr=192.168.1.3:500515455worker1 # [ 118.194476] nix-daemon[852]: reaped child process 854, status = succeeded5456worker2 # [ 118.340888] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:32Z level=info event=scheduler_connected addr=lb:500515457worker2 # [ 118.346092] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5458worker1 # [ 118.370808] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5459lb # [ 118.326989] envoy[893]: conn peer=[2001:db8:1::1]:36656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=255460worker2 # [ 118.547037] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:500515461worker2 # [ 118.552292] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5462worker2 # [ 118.753359] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:500515463worker2 # [ 118.758491] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5464worker1 # [ 118.854130] nix-grpc-daemon[768]: E0000 00:00:1790076813.449390 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5465worker1 # [ 118.860749] nix-grpc-daemon[768]: E0000 00:00:1790076813.449718 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5466worker2 # [ 118.959468] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:500515467worker2 # [ 118.964698] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5468lb # [ 118.332079] envoy[893]: conn peer=[2001:db8:1::1]:36648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2126 ms=19235469lb # [ 118.332237] envoy[893]: conn peer=[2001:db8:1::4]:57598 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=91452 tx=4637 ms=943805470lb # [ 118.332342] envoy[893]: conn peer=[2001:db8:1::1]:35366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=225471lb # [ 118.332458] envoy[893]: conn peer=[2001:db8:1::3]:57748 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=18184 tx=4910 ms=996725472worker1 # [ 119.092477] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5473worker1 # [ 119.099647] systemd[1]: Stopped Nix worker-protocol over gRPC.5474worker1 # [ 119.107782] systemd[1]: nix-grpc-daemon.service: Consumed 5.212s CPU time over 1min 40.715s wall clock time, 70.5M memory peak, 123K incoming IP traffic, 248.1K outgoing IP traffic.5475worker1 # [ 119.123532] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5476worker1 # [ 119.129601] systemd[1]: Closed Nix worker-protocol over gRPC.5477worker1 # [ 119.131464] systemd[1]: nix-grpc-daemon.socket: Consumed 646us CPU time over 1min 48.916s wall clock time, 256K memory peak, 131.9K incoming IP traffic, 113.5K outgoing IP traffic.5478lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5479lb: must succeed: curl -sf localhost:9901/clusters5480worker2 # [ 119.167373] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:500515481worker2 # [ 119.171644] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5483lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5484client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25485worker2 # [ 119.368433] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:500515486worker2 # [ 119.373286] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5487worker2 # [ 119.448980] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570625488worker2 # [ 119.462896] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5489worker2 # [ 119.490905] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)5490worker2 # [ 119.519864] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 2/5)5491worker2 # [ 119.574235] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:500515492worker2 # [ 119.579431] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5493worker2 # [ 119.590729] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 3/5)5494worker2 # [ 119.632887] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 616 ms (attempt 4/5)5495worker2 # [ 119.780347] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:500515496worker2 # [ 119.785096] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker2 # [ 119.986123] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:500515498worker2 # [ 119.991200] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker2 # [ 120.193921] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:500515500worker2 # [ 120.198019] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5501worker2 # [ 120.251495] nix-grpc-daemon[1193]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5502worker2 # [ 120.258582] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5503client # copying 2 paths...5504client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5505worker2 # [ 120.281427] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570625506client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5507worker2 # [ 120.299136] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5508worker2 # [ 120.329338] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=niks3_push_started pid=12535509worker2 # [ 120.396811] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:500515510worker2 # [ 120.400901] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5511worker2 # [ 120.487792] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.074Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5512worker2 # [ 120.592143] nix-daemon[842]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5513worker2 # [ 120.602489] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:500515514worker2 # [ 120.605322] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5515worker2 # [ 120.614004] nix-daemon[842]: reaped child process 1269, status = succeeded5516lb # [ 120.573566] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received uploads request method=POST path=/api/pending_closures5517lb # [ 120.582148] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received uploads request method=POST path=/api/pending_closures5518worker2 # [ 120.639002] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.226Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5519worker2 # [ 120.643497] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.227Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5520worker2 # [ 120.645980] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.227Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5521lb # [ 120.636348] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5522lb # [ 120.655876] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5523lb # [ 120.667227] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5524worker2 # [ 120.722619] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.307Z level=INFO msg="Uploading 3 narinfos"5525lb # [ 120.675788] niks3-server[1033]: 2026/09/22 11:33:35 INFO Signed narinfos id=35 count=15526lb # [ 120.680204] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5527lb # [ 120.688816] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5528lb # [ 120.691661] niks3-server[1033]: 2026/09/22 11:33:35 INFO Signed narinfos id=36 count=25529lb # [ 120.698419] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5530lb # [ 120.706702] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5531lb # [ 120.713988] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5532lb # [ 120.722149] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5533lb # [ 120.729359] niks3-server[1033]: 2026/09/22 11:33:35 INFO Completed upload id=355534worker2 # [ 120.778350] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.365Z level=INFO msg="Upload complete. (286ms)"5535lb # [ 120.730861] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5536lb # [ 120.733945] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5537lb # [ 120.737577] niks3-server[1033]: 2026/09/22 11:33:35 INFO Completed upload id=365538worker2 # [ 120.788375] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=2 nar_bytes_in=3125539worker2 # [ 120.805883] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:500515540worker2 # [ 120.813085] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5541worker2 # [ 120.818601] nix-daemon[842]: reaped child process 1251, status = succeeded5542client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.61 seconds)5543worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5544worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5545worker1 # [ 120.922918] systemd[1]: Listening on Nix worker-protocol over gRPC.5546worker1 # [ 120.933325] systemd[1]: Starting Nix worker-protocol over gRPC...5547worker2 # [ 121.012138] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:500515548worker2 # [ 121.017101] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5549worker1 # [ 121.074996] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5550worker1 # [ 121.077760] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5551worker1 # [ 121.081551] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15552worker1 # [ 121.088071] nix-grpc-daemon[1372]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5553worker1 # [ 121.089500] nix-grpc-daemon[1372]: E0000 00:00:1790076815.682418 1372 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5554lb # [ 121.059502] niks3-server[1033]: 2026/09/22 11:33:35 INFO lead: acquired remote=192.168.1.3:577525555lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5556lb: must succeed: curl -sf localhost:9901/clusters5557worker1 # [ 121.136104] nix-grpc-daemon[1372]: E0000 00:00:1790076815.688703 1372 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5558worker1 # [ 121.138419] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5559worker1 # [ 121.144731] nix-grpc-daemon[1372]: E0000 00:00:1790076815.699345 1414 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5560worker1 # [ 121.147539] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=scheduler_take_over5561worker1 # [ 121.148892] nix-grpc-daemon[1372]: E0000 00:00:1790076815.720161 1414 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5562worker1 # [ 121.151211] nix-grpc-daemon[1372]: E0000 00:00:1790076815.720301 1414 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5563worker1 # [ 121.153696] systemd[1]: Started Nix worker-protocol over gRPC.5564worker1 # [ 121.154598] nix-grpc-daemon[1372]: E0000 00:00:1790076815.739652 1414 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5565worker1 # [ 121.161933] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:500515566worker1 # [ 121.166959] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:597245567worker1 # [ 121.168843] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05568lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5569lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5570(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.82 seconds)5571subtest: a niks3 restart does not cost the leader its role5572lb: must succeed: curl -sf localhost:9901/clusters5573worker2 # [ 121.219318] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:500515574worker1 # [ 121.223407] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=oidc_jwks provider=mock keys=15575worker1 # [ 121.226767] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:597245576worker1 # [ 121.230730] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05577lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5578lb: must succeed: curl -sf localhost:9901/clusters5579lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5580lb # [ 121.280734] systemd[1]: Stopping niks3 server...5581worker1 # [ 121.325556] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=scheduler_lock_lost status=2005582worker2 # [ 121.330343] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_lock_lost status=2005583lb # [ 121.285082] niks3-server[1033]: 2026/09/22 11:33:35 INFO Shutdown signal received, draining in-flight requests timeout=10s5584lb # [ 121.286677] niks3-server[1033]: 2026/09/22 11:33:35 INFO lead: released remote=192.168.1.3:577525585lb # [ 125.961685] systemd[1]: niks3.service: Deactivated successfully.5586lb # [ 125.966800] systemd[1]: Stopped niks3 server.5587lb # [ 125.971282] systemd[1]: niks3.service: Consumed 687ms CPU time over 1min 34.170s wall clock time, 27.6M memory peak, 99K incoming IP traffic, 141.7K outgoing IP traffic.5588lb # [ 125.996493] systemd[1]: Starting niks3 server...5589lb # [ 126.108564] niks3-server[1169]: 2026/09/22 11:33:40 goose: no migrations to run. current version: 202609200000005590lb # [ 126.117174] niks3-server[1169]: 2026/09/22 11:33:40 OK 1_commit_pending_closure.sql (6.64ms)5591lb # [ 126.121155] niks3-server[1169]: 2026/09/22 11:33:40 OK 2_object_stats_trigger.sql (3.57ms)5592lb # [ 126.122454] niks3-server[1169]: 2026/09/22 11:33:40 goose: up to current file version: 25593lb # [ 126.124708] niks3-server[1169]: 2026/09/22 11:33:40 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5594lb # [ 126.136975] niks3-server[1169]: 2026/09/22 11:33:40 INFO Read proxy enabled — serving cache objects from S35595lb # [ 126.145172] niks3-server[1169]: 2026/09/22 11:33:40 INFO Using socket-activated listener address=0.0.0.0:57515596lb # [ 126.147223] niks3-server[1169]: 2026/09/22 11:33:40 INFO systemd watchdog enabled interval=15s5597lb # [ 126.150709] niks3-server[1169]: 2026/09/22 11:33:40 INFO Starting HTTP server address=0.0.0.0:57515598lb: waiting for unit niks3.service5599lb # [ 126.153020] niks3-server[1169]: 2026/09/22 11:33:40 INFO lead: acquired remote=192.168.1.3:577665600lb # [ 126.154252] systemd[1]: Started niks3 server.5601lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5602lb: must succeed: curl -sf localhost:9901/clusters5603lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5604worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5605worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5606lb: must succeed: curl -sf localhost:9901/clusters5607lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5608lb: must succeed: curl -sf localhost:9901/clusters5609lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5610worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5611worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5612client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25613worker1 # [ 126.703743] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:597365614worker1 # [ 126.717948] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5615client # don't know how to build these paths:5616client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5617worker2 # [ 126.748959] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570625618worker1 # [ 126.792807] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05619worker1 # [ 126.799481] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=15620worker1 # [ 126.845537] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597365621worker1 # [ 126.862305] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5)5622worker1 # [ 126.913369] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 2/5)5623worker1 # [ 126.997261] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 104 ms (attempt 3/5)5624worker1 # [ 127.103317] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 244 ms (attempt 4/5)5625worker1 # [ 127.349247] nix-grpc-daemon[1372]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5626worker1 # [ 127.356688] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5627worker1 # [ 127.374617] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:597365628client # copying 1 paths to 'lb:50051'...5629client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5630worker1 # [ 127.390244] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:597365631worker1 # [ 127.404354] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5632worker1 # [ 127.433412] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:42Z level=info event=niks3_push_started pid=14445633worker1 # [ 127.591715] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.186Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5634worker1 # [ 127.696742] nix-daemon[852]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5635worker1 # [ 127.713966] nix-daemon[852]: reaped child process 1459, status = succeeded5636worker1 # [ 127.814707] nix-daemon[852]: accepted connection from pid 1461, user nix-grpc-daemon (trusted)5637worker1 # [ 127.832790] nix-daemon[852]: reaped child process 1464, status = succeeded5638lb # [ 127.787569] niks3-server[1169]: 2026/09/22 11:33:42 INFO Received uploads request method=POST path=/api/pending_closures5639worker1 # [ 127.847992] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.442Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5640worker1 # [ 127.849499] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.442Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5641lb # [ 127.850100] niks3-server[1169]: 2026/09/22 11:33:42 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5642lb # [ 127.863546] niks3-server[1169]: 2026/09/22 11:33:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5643worker1 # [ 127.920822] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.515Z level=INFO msg="Uploading 1 narinfos"5644lb # [ 127.879925] niks3-server[1169]: 2026/09/22 11:33:42 INFO Signed narinfos id=37 count=15645lb # [ 127.894666] niks3-server[1169]: 2026/09/22 11:33:42 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5646lb # [ 127.910967] niks3-server[1169]: 2026/09/22 11:33:42 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5647lb # [ 127.924656] niks3-server[1169]: 2026/09/22 11:33:42 INFO Completed upload id=375648worker1 # [ 127.970102] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.565Z level=INFO msg="Upload complete. (374ms)"5649lb # [ 127.938078] niks3-server[1169]: 2026/09/22 11:33:42 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5650worker1 # [ 127.982148] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=0 paths=1 nar_bytes_in=5445651worker1 # [ 128.002468] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597365652worker1 # [ 128.010938] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5653worker1 # [ 128.018564] nix-daemon[852]: reaped child process 1442, status = succeeded5654client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5655worker1 # [ 128.030542] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5656client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5657lb # [ 128.325701] envoy[893]: conn peer=[2001:db8:1::1]:35382 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=305 ms=13575658client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5659client # farm-a> LOG-farm-a-after-niks3-restart5660worker1 # [ 129.262407] nix-daemon[852]: reaped child process 1468, status = succeeded5661worker1 # [ 129.372068] nix-daemon[852]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)5662worker1 # [ 129.388410] nix-daemon[852]: reaped child process 1487, status = succeeded5663lb # [ 129.350111] niks3-server[1169]: 2026/09/22 11:33:43 INFO Received uploads request method=POST path=/api/pending_closures5664worker1 # [ 129.412293] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.005Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5665worker1 # [ 129.413798] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.005Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5666lb # [ 129.395340] niks3-server[1169]: 2026/09/22 11:33:43 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5667lb # [ 129.408993] niks3-server[1169]: 2026/09/22 11:33:43 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5668worker1 # [ 129.466543] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.061Z level=INFO msg="Uploading 1 narinfos"5669lb # [ 129.421236] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5670lb # [ 129.429581] niks3-server[1169]: 2026/09/22 11:33:44 INFO Signed narinfos id=38 count=15671lb # [ 129.434722] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5672lb # [ 129.444735] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5673lb # [ 129.453803] niks3-server[1169]: 2026/09/22 11:33:44 INFO Completed upload id=385674worker1 # [ 129.498532] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.094Z level=INFO msg="Upload complete. (255ms)"5675lb # [ 129.457169] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5676worker1 # [ 129.520607] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15677worker1 # [ 129.536358] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05678worker1 # [ 129.539913] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=15679worker1 # [ 129.545071] nix-daemon[852]: reaped child process 1466, status = succeeded5680worker1 # [ 129.565333] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597365681worker1 # [ 129.581066] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:597365682worker1 # [ 129.596435] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.191Z level=INFO msg="All 1 paths already cached"5683client # copying 1 paths to 'lb:50051'...5684client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5685worker1 # [ 129.606581] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:597365686worker1 # [ 129.620187] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5687worker1 # [ 129.772126] nix-daemon[852]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5688worker1 # [ 129.789508] nix-daemon[852]: reaped child process 1496, status = succeeded5689worker1 # [ 129.890703] nix-daemon[852]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)5690worker1 # [ 129.905962] nix-daemon[852]: reaped child process 1501, status = succeeded5691lb # [ 129.867130] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received uploads request method=POST path=/api/pending_closures5692worker1 # [ 129.924203] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.519Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5693worker1 # [ 129.925657] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.519Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5694lb # [ 129.897458] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5695lb # [ 129.910316] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5696worker1 # [ 129.964991] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.560Z level=INFO msg="Uploading 1 narinfos"5697lb # [ 129.924592] niks3-server[1169]: 2026/09/22 11:33:44 INFO Signed narinfos id=39 count=15698lb # [ 129.927149] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5699lb # [ 129.939546] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5700lb # [ 129.947849] niks3-server[1169]: 2026/09/22 11:33:44 INFO Completed upload id=395701worker1 # [ 129.992566] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.588Z level=INFO msg="Upload complete. (356ms)"5702lb # [ 129.951523] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5703worker1 # [ 130.004258] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=0 paths=1 nar_bytes_in=6565704worker1 # [ 130.014751] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597365705worker1 # [ 130.023956] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5706worker1 # [ 130.026511] nix-daemon[852]: reaped child process 1491, status = succeeded5707client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5708client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5709worker1 # [ 130.041041] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5710client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5711client # farm-b> LOG-farm-b-after-niks3-restart5712worker1 # [ 131.085366] nix-daemon[852]: reaped child process 1505, status = succeeded5713worker1 # [ 131.199171] nix-daemon[852]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)5714worker1 # [ 131.216874] nix-daemon[852]: reaped child process 1524, status = succeeded5715lb # [ 131.176578] niks3-server[1169]: 2026/09/22 11:33:45 INFO Received uploads request method=POST path=/api/pending_closures5716worker1 # [ 131.232370] nix-grpc-daemon[1444]: time=2026-09-22T11:33:45.827Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5717worker1 # [ 131.233860] nix-grpc-daemon[1444]: time=2026-09-22T11:33:45.827Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5718lb # [ 131.218914] niks3-server[1169]: 2026/09/22 11:33:45 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5719lb # [ 131.231696] niks3-server[1169]: 2026/09/22 11:33:45 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5720lb # [ 131.239332] niks3-server[1169]: 2026/09/22 11:33:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5721worker1 # [ 131.289955] nix-grpc-daemon[1444]: time=2026-09-22T11:33:45.885Z level=INFO msg="Uploading 1 narinfos"5722lb # [ 131.248441] niks3-server[1169]: 2026/09/22 11:33:45 INFO Signed narinfos id=40 count=15723lb # [ 131.255736] niks3-server[1169]: 2026/09/22 11:33:45 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5724lb # [ 131.268074] niks3-server[1169]: 2026/09/22 11:33:45 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5725lb # [ 131.274805] niks3-server[1169]: 2026/09/22 11:33:45 INFO Completed upload id=405726worker1 # [ 131.319434] nix-grpc-daemon[1444]: time=2026-09-22T11:33:45.915Z level=INFO msg="Upload complete. (268ms)"5727lb # [ 131.278477] niks3-server[1169]: 2026/09/22 11:33:45 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5728worker1 # [ 131.338107] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15729worker1 # [ 131.356319] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05730worker1 # [ 131.360677] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=15731worker1 # [ 131.365109] nix-daemon[852]: reaped child process 1503, status = succeeded5732worker1 # [ 131.381282] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597365733worker1 # [ 131.396277] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:597365734worker1 # [ 131.411972] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.007Z level=INFO msg="All 2 paths already cached"5735client # copying 1 paths to 'lb:50051'...5736client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5737worker1 # [ 131.423644] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:597365738worker1 # [ 131.437570] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5739worker1 # [ 131.588085] nix-daemon[852]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)5740worker1 # [ 131.611955] nix-daemon[852]: reaped child process 1533, status = succeeded5741worker1 # [ 131.703279] nix-daemon[852]: accepted connection from pid 1535, user nix-grpc-daemon (trusted)5742worker1 # [ 131.721564] nix-daemon[852]: reaped child process 1538, status = succeeded5743lb # [ 131.674639] niks3-server[1169]: 2026/09/22 11:33:46 INFO Received uploads request method=POST path=/api/pending_closures5744worker1 # [ 131.728832] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.324Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5745worker1 # [ 131.730294] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.324Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5746lb # [ 131.701109] niks3-server[1169]: 2026/09/22 11:33:46 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5747worker1 # [ 131.758024] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.353Z level=INFO msg="Uploading 1 narinfos"5748lb # [ 131.714599] niks3-server[1169]: 2026/09/22 11:33:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5749lb # [ 131.726360] niks3-server[1169]: 2026/09/22 11:33:46 INFO Signed narinfos id=41 count=15750lb # [ 131.733074] niks3-server[1169]: 2026/09/22 11:33:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5751lb # [ 131.743462] niks3-server[1169]: 2026/09/22 11:33:46 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5752lb # [ 131.750812] niks3-server[1169]: 2026/09/22 11:33:46 INFO Completed upload id=415753worker1 # [ 131.794819] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.390Z level=INFO msg="Upload complete. (341ms)"5754lb # [ 131.752968] niks3-server[1169]: 2026/09/22 11:33:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5755worker1 # [ 131.804509] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=0 paths=1 nar_bytes_in=7845756worker1 # [ 131.816186] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597365757worker1 # [ 131.823957] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5758worker1 # [ 131.832181] nix-daemon[852]: reaped child process 1528, status = succeeded5759worker1 # [ 131.837951] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5760client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5761client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5762client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5763client # farm-top> LOG-farm-top-after-niks3-restart5764worker1 # [ 133.268138] nix-daemon[852]: reaped child process 1542, status = succeeded5765worker1 # [ 133.367021] nix-daemon[852]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)5766worker1 # [ 133.385717] nix-daemon[852]: reaped child process 1561, status = succeeded5767lb # [ 133.346256] niks3-server[1169]: 2026/09/22 11:33:47 INFO Received uploads request method=POST path=/api/pending_closures5768worker1 # [ 133.403635] nix-grpc-daemon[1444]: time=2026-09-22T11:33:47.997Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5769worker1 # [ 133.405254] nix-grpc-daemon[1444]: time=2026-09-22T11:33:47.997Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5770lb # [ 133.377605] niks3-server[1169]: 2026/09/22 11:33:47 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5771lb # [ 133.399640] niks3-server[1169]: 2026/09/22 11:33:47 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5772worker1 # [ 133.453044] nix-grpc-daemon[1444]: time=2026-09-22T11:33:48.048Z level=INFO msg="Uploading 1 narinfos"5773lb # [ 133.410841] niks3-server[1169]: 2026/09/22 11:33:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5774lb # [ 133.419030] niks3-server[1169]: 2026/09/22 11:33:47 INFO Signed narinfos id=42 count=15775lb # [ 133.423699] niks3-server[1169]: 2026/09/22 11:33:48 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5776lb # [ 133.431173] niks3-server[1169]: 2026/09/22 11:33:48 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5777lb # [ 133.436789] niks3-server[1169]: 2026/09/22 11:33:48 INFO Completed upload id=425778worker1 # [ 133.481439] nix-grpc-daemon[1444]: time=2026-09-22T11:33:48.077Z level=INFO msg="Upload complete. (231ms)"5779lb # [ 133.439438] niks3-server[1169]: 2026/09/22 11:33:48 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5780worker1 # [ 133.493089] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15781worker1 # [ 133.512248] nix-daemon[852]: reaped child process 1540, status = succeeded5782client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 7.07 seconds)5783client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5784client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5785(finished: subtest: a niks3 restart does not cost the leader its role, in 12.51 seconds)5786subtest: scheduler down: the other node takes the lock and keeps it5787lb: must succeed: curl -sf localhost:9901/clusters5788lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5789lb: must succeed: curl -sf localhost:9901/clusters5790lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5791lb: must succeed: curl -sf localhost:9901/clusters5792lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5793lb: must succeed: curl -sf localhost:9901/clusters5794lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5795worker1 # [ 133.995656] systemd[1]: Stopping Nix worker-protocol over gRPC...5796worker1 # [ 134.118041] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:48Z level=info event=signal_exit5797lb # [ 134.148203] niks3-server[1169]: 2026/09/22 11:33:48 INFO lead: released remote=192.168.1.3:577665798worker2 # [ 135.199106] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:49Z level=info event=scheduler_restarting addr=lb:500515799worker1 # [ 135.199088] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=scheduler_restarting5800worker1 # [ 135.204443] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=scheduler_restarting addr=lb:500515801worker1 # [ 135.208867] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:597245802worker1 # [ 135.213830] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=worker_gone addr=192.168.1.4:500515803worker1 # [ 135.218390] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:597245804worker1 # [ 135.228164] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=worker_gone addr=192.168.1.3:500515805worker1 # [ 135.246617] nix-daemon[852]: reaped child process 1434, status = succeeded5806worker2 # [ 135.405095] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:49Z level=info event=scheduler_connected addr=lb:500515807worker2 # [ 135.410521] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5808worker1 # [ 135.414357] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5809worker2 # [ 135.611542] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_connected addr=lb:500515810worker2 # [ 135.616823] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5811worker2 # [ 135.817665] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_connected addr=lb:500515812worker2 # [ 135.822567] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5813worker2 # [ 136.023631] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_connected addr=lb:500515814worker2 # [ 136.028927] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5815worker1 # [ 136.150882] nix-grpc-daemon[1372]: E0000 00:00:1790076830.746189 1417 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5816worker1 # [ 136.151206] nix-grpc-daemon[1372]: E0000 00:00:1790076830.746473 1417 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5817worker2 # [ 136.191842] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_take_over5818lb # [ 136.143698] niks3-server[1169]: 2026/09/22 11:33:50 INFO lead: acquired remote=192.168.1.4:494305819worker2 # [ 136.231468] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_connected addr=lb:500515820worker2 # [ 136.235667] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5821worker1 # [ 136.355980] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5822worker1 # [ 136.363279] systemd[1]: Stopped Nix worker-protocol over gRPC.5823worker1 # [ 136.369399] systemd[1]: nix-grpc-daemon.service: Consumed 1.415s CPU time over 15.425s wall clock time, 38.2M memory peak, 35.4K incoming IP traffic, 56.4K outgoing IP traffic.5824worker1 # [ 136.388889] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5825worker1 # [ 136.391214] systemd[1]: Closed Nix worker-protocol over gRPC.5826worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5827worker1 # [ 136.397003] systemd[1]: nix-grpc-daemon.socket: Consumed 984us CPU time over 15.471s wall clock time, 256K memory peak, 36.8K incoming IP traffic, 27.8K outgoing IP traffic.5828worker2 # [ 136.432474] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=scheduler_connected addr=lb:500515829worker2 # [ 136.442276] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5830worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5831worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5832worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5833worker2 # [ 136.640121] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=scheduler_connected addr=lb:500515834worker2 # [ 136.667078] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=oidc_jwks provider=mock keys=15835worker2 # [ 136.671753] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:525865836worker2 # [ 136.677347] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05837worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5838worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5839client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25840client # don't know how to build these paths:5841client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5842worker2 # [ 137.820583] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570625843worker2 # [ 137.827121] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570625844worker2 # [ 137.853223] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05845worker2 # [ 137.860360] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15846worker2 # [ 137.901288] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570625847worker2 # [ 137.916890] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570625848client # copying 1 paths to 'lb:50051'...5849client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5850worker2 # [ 137.933205] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570625851worker2 # [ 137.949935] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5852worker2 # [ 138.097087] nix-daemon[842]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5853worker2 # [ 138.115333] nix-daemon[842]: reaped child process 1295, status = succeeded5854worker2 # [ 138.216563] nix-daemon[842]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5855worker2 # [ 138.231416] nix-daemon[842]: reaped child process 1300, status = succeeded5856lb # [ 138.191299] niks3-server[1169]: 2026/09/22 11:33:52 INFO Received uploads request method=POST path=/api/pending_closures5857worker2 # [ 138.251591] nix-grpc-daemon[1253]: time=2026-09-22T11:33:52.837Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5858worker2 # [ 138.253558] nix-grpc-daemon[1253]: time=2026-09-22T11:33:52.837Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5859lb # [ 138.227643] niks3-server[1169]: 2026/09/22 11:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5860worker2 # [ 138.286679] nix-grpc-daemon[1253]: time=2026-09-22T11:33:52.873Z level=INFO msg="Uploading 1 narinfos"5861lb # [ 138.242018] niks3-server[1169]: 2026/09/22 11:33:52 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5862lb # [ 138.255409] niks3-server[1169]: 2026/09/22 11:33:52 INFO Signed narinfos id=43 count=15863worker2 # [ 138.310534] nix-grpc-daemon[1253]: time=2026-09-22T11:33:52.897Z level=INFO msg="Upload complete. (345ms)"5864lb # [ 138.263040] niks3-server[1169]: 2026/09/22 11:33:52 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5865lb # [ 138.270131] niks3-server[1169]: 2026/09/22 11:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5866lb # [ 138.274117] niks3-server[1169]: 2026/09/22 11:33:52 INFO Completed upload id=435867worker2 # [ 138.320983] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=1 nar_bytes_in=5205868lb # [ 138.277068] niks3-server[1169]: 2026/09/22 11:33:52 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5869worker2 # [ 138.336288] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570625870worker2 # [ 138.347548] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5871worker2 # [ 138.351764] nix-daemon[842]: reaped child process 1290, status = succeeded5872client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5873worker2 # [ 138.363838] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5874client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5875lb # [ 138.329380] envoy[893]: conn peer=[2001:db8:1::1]:46662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9766 tx=4485 ms=68095876lb # [ 138.334452] envoy[893]: conn peer=[2001:db8:1::3]:49820 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1065 tx=425 ms=142475877client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5878client # farm-a> LOG-farm-a-on-standby5879worker2 # [ 139.220854] nix-daemon[842]: reaped child process 1304, status = succeeded5880worker2 # [ 139.325685] nix-daemon[842]: accepted connection from pid 1320, user nix-grpc-daemon (trusted)5881worker2 # [ 139.343935] nix-daemon[842]: reaped child process 1323, status = succeeded5882lb # [ 139.295119] niks3-server[1169]: 2026/09/22 11:33:53 INFO Received uploads request method=POST path=/api/pending_closures5883worker2 # [ 139.356376] nix-grpc-daemon[1253]: time=2026-09-22T11:33:53.942Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5884worker2 # [ 139.357941] nix-grpc-daemon[1253]: time=2026-09-22T11:33:53.942Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5885lb # [ 139.342765] niks3-server[1169]: 2026/09/22 11:33:53 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5886worker2 # [ 139.404731] nix-grpc-daemon[1253]: time=2026-09-22T11:33:53.990Z level=INFO msg="Uploading 1 narinfos"5887lb # [ 139.358332] niks3-server[1169]: 2026/09/22 11:33:53 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5888lb # [ 139.367002] niks3-server[1169]: 2026/09/22 11:33:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5889lb # [ 139.377725] niks3-server[1169]: 2026/09/22 11:33:53 INFO Signed narinfos id=44 count=15890lb # [ 139.379598] niks3-server[1169]: 2026/09/22 11:33:53 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5891lb # [ 139.386051] niks3-server[1169]: 2026/09/22 11:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5892worker2 # [ 139.435205] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.022Z level=INFO msg="Upload complete. (257ms)"5893lb # [ 139.388999] niks3-server[1169]: 2026/09/22 11:33:53 INFO Completed upload id=445894lb # [ 139.391858] niks3-server[1169]: 2026/09/22 11:33:53 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5895worker2 # [ 139.454134] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15896worker2 # [ 139.475184] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05897worker2 # [ 139.481384] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15898worker2 # [ 139.486156] nix-daemon[842]: reaped child process 1302, status = succeeded5899worker2 # [ 139.513552] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570625900worker2 # [ 139.529067] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570625901worker2 # [ 139.542894] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.129Z level=INFO msg="All 1 paths already cached"5902client # copying 1 paths to 'lb:50051'...5903client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5904worker2 # [ 139.553469] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570625905worker2 # [ 139.566856] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5906worker2 # [ 139.718627] nix-daemon[842]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5907worker2 # [ 139.734586] nix-daemon[842]: reaped child process 1332, status = succeeded5908worker2 # [ 139.836342] nix-daemon[842]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5909worker2 # [ 139.851160] nix-daemon[842]: reaped child process 1337, status = succeeded5910lb # [ 139.809056] niks3-server[1169]: 2026/09/22 11:33:54 INFO Received uploads request method=POST path=/api/pending_closures5911worker2 # [ 139.867223] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.453Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5912worker2 # [ 139.868987] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.454Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5913lb # [ 139.839741] niks3-server[1169]: 2026/09/22 11:33:54 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5914worker2 # [ 139.899653] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.486Z level=INFO msg="Uploading 1 narinfos"5915lb # [ 139.854329] niks3-server[1169]: 2026/09/22 11:33:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5916lb # [ 139.865583] niks3-server[1169]: 2026/09/22 11:33:54 INFO Signed narinfos id=45 count=15917lb # [ 139.870808] niks3-server[1169]: 2026/09/22 11:33:54 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5918lb # [ 139.882154] niks3-server[1169]: 2026/09/22 11:33:54 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5919lb # [ 139.888866] niks3-server[1169]: 2026/09/22 11:33:54 INFO Completed upload id=455920worker2 # [ 139.937829] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.524Z level=INFO msg="Upload complete. (355ms)"5921lb # [ 139.892685] niks3-server[1169]: 2026/09/22 11:33:54 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5922worker2 # [ 139.948278] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=1 nar_bytes_in=6405923worker2 # [ 139.958478] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570625924worker2 # [ 139.967934] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5925worker2 # [ 139.971085] nix-daemon[842]: reaped child process 1327, status = succeeded5926client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5927worker2 # [ 139.984474] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5928client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5929client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5930client # farm-b> LOG-farm-b-on-standby5931worker2 # [ 141.064158] nix-daemon[842]: reaped child process 1341, status = succeeded5932worker2 # [ 141.163918] nix-daemon[842]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5933worker2 # [ 141.184534] nix-daemon[842]: reaped child process 1360, status = succeeded5934lb # [ 141.135158] niks3-server[1169]: 2026/09/22 11:33:55 INFO Received uploads request method=POST path=/api/pending_closures5935worker2 # [ 141.196112] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.781Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5936worker2 # [ 141.198376] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.782Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5937lb # [ 141.175773] niks3-server[1169]: 2026/09/22 11:33:55 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5938worker2 # [ 141.237272] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.824Z level=INFO msg="Uploading 1 narinfos"5939lb # [ 141.190547] niks3-server[1169]: 2026/09/22 11:33:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5940lb # [ 141.200778] niks3-server[1169]: 2026/09/22 11:33:55 INFO Signed narinfos id=46 count=15941lb # [ 141.205661] niks3-server[1169]: 2026/09/22 11:33:55 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5942lb # [ 141.214978] niks3-server[1169]: 2026/09/22 11:33:55 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5943lb # [ 141.219640] niks3-server[1169]: 2026/09/22 11:33:55 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5944worker2 # [ 141.267958] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.854Z level=INFO msg="Upload complete. (273ms)"5945lb # [ 141.222664] niks3-server[1169]: 2026/09/22 11:33:55 INFO Completed upload id=465946lb # [ 141.225078] niks3-server[1169]: 2026/09/22 11:33:55 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5947worker2 # [ 141.286195] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15948worker2 # [ 141.302631] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05949worker2 # [ 141.306649] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15950worker2 # [ 141.311103] nix-daemon[842]: reaped child process 1339, status = succeeded5951worker2 # [ 141.329173] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570625952worker2 # [ 141.340627] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570625953worker2 # [ 141.354972] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.941Z level=INFO msg="All 2 paths already cached"5954client # copying 1 paths to 'lb:50051'...5955client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5956worker2 # [ 141.366234] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570625957worker2 # [ 141.379716] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5958worker2 # [ 141.529057] nix-daemon[842]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5959worker2 # [ 141.547292] nix-daemon[842]: reaped child process 1369, status = succeeded5960worker2 # [ 141.648140] nix-daemon[842]: accepted connection from pid 1371, user nix-grpc-daemon (trusted)5961worker2 # [ 141.662147] nix-daemon[842]: reaped child process 1374, status = succeeded5962lb # [ 141.621036] niks3-server[1169]: 2026/09/22 11:33:56 INFO Received uploads request method=POST path=/api/pending_closures5963worker2 # [ 141.679432] nix-grpc-daemon[1253]: time=2026-09-22T11:33:56.266Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5964worker2 # [ 141.681066] nix-grpc-daemon[1253]: time=2026-09-22T11:33:56.266Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5965lb # [ 141.648675] niks3-server[1169]: 2026/09/22 11:33:56 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5966worker2 # [ 141.710776] nix-grpc-daemon[1253]: time=2026-09-22T11:33:56.297Z level=INFO msg="Uploading 1 narinfos"5967lb # [ 141.661636] niks3-server[1169]: 2026/09/22 11:33:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5968lb # [ 141.671408] niks3-server[1169]: 2026/09/22 11:33:56 INFO Signed narinfos id=47 count=15969lb # [ 141.679892] niks3-server[1169]: 2026/09/22 11:33:56 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5970lb # [ 141.692607] niks3-server[1169]: 2026/09/22 11:33:56 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5971worker2 # [ 141.749245] nix-grpc-daemon[1253]: time=2026-09-22T11:33:56.336Z level=INFO msg="Upload complete. (354ms)"5972lb # [ 141.702472] niks3-server[1169]: 2026/09/22 11:33:56 INFO Completed upload id=475973lb # [ 141.705828] niks3-server[1169]: 2026/09/22 11:33:56 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5974worker2 # [ 141.756372] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=1 nar_bytes_in=7685975worker2 # [ 141.770989] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570625976worker2 # [ 141.776953] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5977worker2 # [ 141.781479] nix-daemon[842]: reaped child process 1364, status = succeeded5978client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5979worker2 # [ 141.793549] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5980client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5981client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5982client # farm-top> LOG-farm-top-on-standby5983worker2 # [ 143.095447] nix-daemon[842]: reaped child process 1378, status = succeeded5984worker2 # [ 143.165216] nix-daemon[842]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5985worker2 # [ 143.184913] nix-daemon[842]: reaped child process 1397, status = succeeded5986lb # [ 143.135489] niks3-server[1169]: 2026/09/22 11:33:57 INFO Received uploads request method=POST path=/api/pending_closures5987worker2 # [ 143.195469] nix-grpc-daemon[1253]: time=2026-09-22T11:33:57.780Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5988worker2 # [ 143.197145] nix-grpc-daemon[1253]: time=2026-09-22T11:33:57.780Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5989lb # [ 143.167030] niks3-server[1169]: 2026/09/22 11:33:57 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5990lb # [ 143.181997] niks3-server[1169]: 2026/09/22 11:33:57 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5991worker2 # [ 143.244743] nix-grpc-daemon[1253]: time=2026-09-22T11:33:57.831Z level=INFO msg="Uploading 1 narinfos"5992lb # [ 143.197214] niks3-server[1169]: 2026/09/22 11:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5993lb # [ 143.204733] niks3-server[1169]: 2026/09/22 11:33:57 INFO Signed narinfos id=48 count=15994lb # [ 143.209830] niks3-server[1169]: 2026/09/22 11:33:57 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5995lb # [ 143.217356] niks3-server[1169]: 2026/09/22 11:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5996worker2 # [ 143.270147] nix-grpc-daemon[1253]: time=2026-09-22T11:33:57.857Z level=INFO msg="Upload complete. (266ms)"5997lb # [ 143.223544] niks3-server[1169]: 2026/09/22 11:33:57 INFO Completed upload id=485998lb # [ 143.226592] niks3-server[1169]: 2026/09/22 11:33:57 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5999worker2 # [ 143.285102] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=16000worker2 # [ 143.299658] nix-daemon[842]: reaped child process 1376, status = succeeded6001client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.73 seconds)6002client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath6003client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds)6004worker1 # [ 143.587225] systemd[1]: Listening on Nix worker-protocol over gRPC.6005worker1 # [ 143.598164] systemd[1]: Starting Nix worker-protocol over gRPC...6006worker1 # [ 143.738690] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6007worker1 # [ 143.740978] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6008worker1 # [ 143.744760] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16009worker1 # [ 143.751240] nix-grpc-daemon[1579]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6010lb: must succeed: curl -sf localhost:9901/clusters6011worker1 # [ 143.765910] nix-grpc-daemon[1579]: E0000 00:00:1790076838.344712 1579 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6012worker1 # [ 143.792244] nix-grpc-daemon[1579]: E0000 00:00:1790076838.346778 1579 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6013worker1 # [ 143.798858] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6014worker1 # [ 143.801096] nix-grpc-daemon[1579]: E0000 00:00:1790076838.356910 1620 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6015worker1 # [ 143.803552] nix-grpc-daemon[1579]: E0000 00:00:1790076838.374810 1620 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6016worker1 # [ 143.805872] nix-grpc-daemon[1579]: E0000 00:00:1790076838.375041 1620 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6017worker1 # [ 143.808187] nix-grpc-daemon[1579]: E0000 00:00:1790076838.393460 1620 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6018worker1 # [ 143.813369] systemd[1]: Started Nix worker-protocol over gRPC.6019worker1 # [ 143.815900] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=scheduler_connected addr=lb:500516020worker2 # [ 143.821196] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:525866021worker2 # [ 143.826273] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06022lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6023worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6024worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6025lb: must succeed: curl -sf localhost:9901/clusters6026lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6027lb: must succeed: curl -sf localhost:9901/clusters6028lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6029worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6030worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6031(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.38 seconds)6032subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6033client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs6034client # [ 144.181143] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.6035client # Running as unit: rs.service; invocation ID: 39fc12e5b1b44548a33753522390304c6036client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)6037client # [ 144.381657] nix[1339]: don't know how to build these paths:6038client # [ 144.383331] nix[1339]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv6039worker2 # [ 144.361315] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570626040worker2 # [ 144.375182] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=06041worker2 # [ 144.381908] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=16042worker2 # [ 144.421940] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570626043worker2 # [ 144.438172] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570626044client # [ 144.471812] nix[1339]: copying 1 paths to 'lb:50051'...6045worker2 # [ 144.452789] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570626046client # [ 144.474481] nix[1339]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...6047worker2 # [ 144.467479] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)6048worker2 # [ 144.625682] nix-daemon[842]: accepted connection from pid 1410, user nix-grpc-daemon (trusted)6049worker2 # [ 144.652767] nix-daemon[842]: reaped child process 1413, status = succeeded6050worker2 # [ 144.741112] nix-daemon[842]: accepted connection from pid 1415, user nix-grpc-daemon (trusted)6051worker2 # [ 144.757483] nix-daemon[842]: reaped child process 1418, status = succeeded6052lb # [ 144.716701] niks3-server[1169]: 2026/09/22 11:33:59 INFO Received uploads request method=POST path=/api/pending_closures6053worker2 # [ 144.775364] nix-grpc-daemon[1253]: time=2026-09-22T11:33:59.361Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6054worker2 # [ 144.777038] nix-grpc-daemon[1253]: time=2026-09-22T11:33:59.361Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"6055worker2 # [ 144.810772] nix-grpc-daemon[1253]: time=2026-09-22T11:33:59.397Z level=INFO msg="Uploading 1 narinfos"6056lb # [ 144.750779] niks3-server[1169]: 2026/09/22 11:33:59 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst6057lb # [ 144.765897] niks3-server[1169]: 2026/09/22 11:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6058lb # [ 144.774664] niks3-server[1169]: 2026/09/22 11:33:59 INFO Signed narinfos id=49 count=16059lb # [ 144.782384] niks3-server[1169]: 2026/09/22 11:33:59 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls6060worker2 # [ 144.835517] nix-grpc-daemon[1253]: time=2026-09-22T11:33:59.422Z level=INFO msg="Upload complete. (349ms)"6061lb # [ 144.790598] niks3-server[1169]: 2026/09/22 11:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6062lb # [ 144.795865] niks3-server[1169]: 2026/09/22 11:33:59 INFO Completed upload id=496063worker2 # [ 144.844235] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=1 nar_bytes_in=4166064lb # [ 144.799580] niks3-server[1169]: 2026/09/22 11:33:59 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo6065worker2 # [ 144.854919] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570626066worker2 # [ 144.861613] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)6067worker2 # [ 144.866581] nix-daemon[842]: reaped child process 1408, status = succeeded6068client # [ 144.895765] nix[1339]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...6069worker2 # [ 144.878086] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)6070lb: must succeed: curl -sf localhost:9901/clusters6071lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6072lb: must succeed: curl -sf localhost:9901/clusters6073lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6074lb: must succeed: curl -sf localhost:9901/clusters6075lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6076lb: must succeed: curl -sf localhost:9901/clusters6077lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6078worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6079worker1 # [ 146.952843] systemd-journald[270]: Received client request to rotate journal, rotating.6080worker1 # [ 146.971834] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/b4b129e142a041e1939b13b11d6a8e9d.6081worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds)6082worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6083worker2 # [ 147.022433] systemd[1]: Stopping Nix worker-protocol over gRPC...6084worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)6085client: must succeed: sleep 5.06086worker2 # [ 147.422988] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:02Z level=info event=draining6087lb # [ 148.330137] envoy[893]: conn peer=[2001:db8:1::1]:59030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9619 tx=4418 ms=54766088client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6089worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6090worker2 # [ 152.587541] nix-daemon[842]: reaped child process 1422, status = succeeded6091worker2 # [ 152.667994] nix-daemon[842]: accepted connection from pid 1463, user nix-grpc-daemon (trusted)6092worker2 # [ 152.687396] nix-daemon[842]: reaped child process 1466, status = succeeded6093lb # [ 152.643182] niks3-server[1169]: 2026/09/22 11:34:07 INFO Received uploads request method=POST path=/api/pending_closures6094worker2 # [ 152.704465] nix-grpc-daemon[1253]: time=2026-09-22T11:34:07.288Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6095worker2 # [ 152.706159] nix-grpc-daemon[1253]: time=2026-09-22T11:34:07.288Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"6096lb # [ 152.676330] niks3-server[1169]: 2026/09/22 11:34:07 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv6097lb # [ 152.693827] niks3-server[1169]: 2026/09/22 11:34:07 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6098worker2 # [ 152.755519] nix-grpc-daemon[1253]: time=2026-09-22T11:34:07.342Z level=INFO msg="Uploading 1 narinfos"6099lb # [ 152.706863] niks3-server[1169]: 2026/09/22 11:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6100lb # [ 152.715479] niks3-server[1169]: 2026/09/22 11:34:07 INFO Signed narinfos id=50 count=16101lb # [ 152.720986] niks3-server[1169]: 2026/09/22 11:34:07 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls6102lb # [ 152.729503] niks3-server[1169]: 2026/09/22 11:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6103lb # [ 152.734354] niks3-server[1169]: 2026/09/22 11:34:07 INFO Completed upload id=506104worker2 # [ 152.782540] nix-grpc-daemon[1253]: time=2026-09-22T11:34:07.369Z level=INFO msg="Upload complete. (285ms)"6105lb # [ 152.737398] niks3-server[1169]: 2026/09/22 11:34:07 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo6106worker2 # [ 152.797093] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=16107worker2 # [ 152.813862] nix-daemon[842]: reaped child process 1420, status = succeeded6108client # [ 152.845853] systemd[1]: rs.service: Deactivated successfully.6109client # [ 152.850935] systemd[1]: rs.service: Consumed 190ms CPU time over 8.665s wall clock time, 41.2M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic.6110worker2 # [ 153.425130] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:08Z level=info event=signal_exit6111lb # [ 154.149847] niks3-server[1169]: 2026/09/22 11:34:08 INFO lead: released remote=192.168.1.4:494306112worker1 # [ 155.195113] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:09Z level=info event=scheduler_restarting addr=lb:500516113worker2 # [ 155.202354] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=scheduler_restarting6114worker2 # [ 155.207665] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=scheduler_restarting addr=lb:500516115worker2 # [ 155.212421] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:525866116worker2 # [ 155.217479] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=worker_gone addr=192.168.1.3:500516117worker2 # [ 155.223064] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:525866118worker2 # [ 155.228248] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=worker_gone addr=192.168.1.4:500516119worker2 # [ 155.251372] nix-daemon[842]: reaped child process 1245, status = succeeded6120worker1 # [ 155.394723] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:09Z level=info event=scheduler_connected addr=lb:500516121worker1 # [ 155.400282] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6122worker2 # [ 155.419276] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6123worker2 # [ 155.508183] nix-grpc-daemon[1193]: E0000 00:00:1790076850.094817 1238 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6124worker2 # [ 155.514794] nix-grpc-daemon[1193]: E0000 00:00:1790076850.095117 1238 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6125worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.26 seconds)6126client: waiting for success: ! systemctl is-active rs6127worker1 # [ 155.601096] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:500516128worker1 # [ 155.606281] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6129client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)6130client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6131client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)6132client: must fail: journalctl -u rs | grep -q 'reconnecting'6133client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6134lb: must succeed: curl -sf localhost:9901/clusters6135lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6136worker1 # [ 155.807034] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:500516137worker1 # [ 155.811879] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6138worker2 # [ 155.966821] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6139worker2 # [ 155.975080] systemd[1]: Stopped Nix worker-protocol over gRPC.6140worker2 # [ 155.980648] systemd[1]: nix-grpc-daemon.service: Consumed 2.087s CPU time over 39.712s wall clock time, 39.2M memory peak, 59.1K incoming IP traffic, 132.1K outgoing IP traffic.6141worker2 # [ 155.997804] systemd[1]: Starting scheduler-token.service...6142worker1 # [ 156.012689] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:500516143worker1 # [ 156.017772] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6144worker2 # [ 156.074486] systemd[1]: scheduler-token.service: Deactivated successfully.6145worker2 # [ 156.079203] systemd[1]: Finished scheduler-token.service.6146worker2 # [ 156.084813] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6147worker2 # [ 156.091271] systemd[1]: Starting Nix worker-protocol over gRPC...6148worker2 # [ 156.214499] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6149worker2 # [ 156.216760] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6150worker2 # [ 156.220512] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16151worker1 # [ 156.220254] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:500516152worker2 # [ 156.227107] nix-grpc-daemon[1476]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6153worker1 # [ 156.224254] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6154lb # [ 156.186644] niks3-server[1169]: 2026/09/22 11:34:10 INFO lead: acquired remote=192.168.1.4:358726155worker2 # [ 156.242399] nix-grpc-daemon[1476]: E0000 00:00:1790076850.812027 1476 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6156worker2 # [ 156.267700] nix-grpc-daemon[1476]: E0000 00:00:1790076850.814073 1476 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6157worker2 # [ 156.270675] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6158worker2 # [ 156.272533] nix-grpc-daemon[1476]: E0000 00:00:1790076850.821522 1517 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6159worker2 # [ 156.274832] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=scheduler_take_over6160worker2 # [ 156.278962] nix-grpc-daemon[1476]: E0000 00:00:1790076850.839263 1517 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6161worker2 # [ 156.283389] nix-grpc-daemon[1476]: E0000 00:00:1790076850.839406 1517 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6162worker2 # [ 156.285892] nix-grpc-daemon[1476]: E0000 00:00:1790076850.853911 1517 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6163worker2 # [ 156.288153] systemd[1]: Started Nix worker-protocol over gRPC.6164worker2 # [ 156.288953] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:500516165worker2 # [ 156.290207] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6166worker1 # [ 156.420813] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:500516167worker1 # [ 156.425791] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6168worker2 # [ 156.484854] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:500516169worker2 # [ 156.489740] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6170worker1 # [ 156.626422] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:500516171worker1 # [ 156.631562] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6172worker2 # [ 156.690901] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:500516173worker2 # [ 156.696270] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6174lb: must succeed: curl -sf localhost:9901/clusters6175worker1 # [ 156.832449] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:500516176worker1 # [ 156.837549] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6177lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6178worker2 # [ 156.897105] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:500516179worker2 # [ 156.902169] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6180worker1 # [ 157.038422] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:500516181worker2 # [ 157.048939] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:343526182worker2 # [ 157.054271] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06183worker2 # [ 157.103285] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:500516184worker2 # [ 157.116138] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=oidc_jwks provider=mock keys=16185worker2 # [ 157.121163] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:343526186worker2 # [ 157.126460] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06187lb: must succeed: curl -sf localhost:9901/clusters6188lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6189worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6190worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6191(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.90 seconds)6192subtest: scheduler restart mid-build: build finishes, no double build6193client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid6194client # [ 158.089317] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.6195client # Running as unit: mid.service; invocation ID: e13b8894d668482296c5d299038723446196client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)6197worker2 # [ 158.286559] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343566198worker2 # [ 158.300884] nix-daemon[842]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)6199client # [ 158.337175] nix[1375]: don't know how to build these paths:6200client # [ 158.338593] nix[1375]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6201worker2 # [ 158.333053] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:12Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06202worker2 # [ 158.339130] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:12Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16203lb # [ 158.336370] envoy[893]: conn peer=[2001:db8:1::1]:50652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1542 ms=84476204lb # [ 158.341505] envoy[893]: conn peer=[2001:db8:1::4]:50414 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=18744 tx=1341 ms=389486205worker1 # [ 158.401207] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483466206worker1 # [ 158.415925] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6207worker1 # [ 158.442310] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 1/5)6208worker1 # [ 158.470358] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 2/5)6209worker1 # [ 158.635267] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 313 ms (attempt 3/5)6210worker1 # [ 158.950108] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 225 ms (attempt 4/5)6211worker1 # [ 159.177323] nix-grpc-daemon[1579]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6212worker1 # [ 159.182975] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6213worker1 # [ 159.201801] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483466214client # [ 159.235712] nix[1375]: copying 1 paths to 'lb:50051'...6215client # [ 159.241496] nix[1375]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...6216worker1 # [ 159.218325] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483466217worker1 # [ 159.231957] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6218worker1 # [ 159.272278] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:13Z level=info event=niks3_push_started pid=17026219worker1 # [ 159.424591] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.019Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6220worker1 # [ 159.530392] nix-daemon[852]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6221worker1 # [ 159.552650] nix-daemon[852]: reaped child process 1718, status = succeeded6222worker1 # [ 159.645794] nix-daemon[852]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6223worker1 # [ 159.662951] nix-daemon[852]: reaped child process 1723, status = succeeded6224lb # [ 159.622008] niks3-server[1169]: 2026/09/22 11:34:14 INFO Received uploads request method=POST path=/api/pending_closures6225worker1 # [ 159.679100] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.273Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6226worker1 # [ 159.680913] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.273Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"6227lb # [ 159.658121] niks3-server[1169]: 2026/09/22 11:34:14 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst6228worker1 # [ 159.718559] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.314Z level=INFO msg="Uploading 1 narinfos"6229lb # [ 159.673654] niks3-server[1169]: 2026/09/22 11:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6230lb # [ 159.684538] niks3-server[1169]: 2026/09/22 11:34:14 INFO Signed narinfos id=51 count=16231lb # [ 159.691220] niks3-server[1169]: 2026/09/22 11:34:14 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls6232lb # [ 159.705492] niks3-server[1169]: 2026/09/22 11:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6233lb # [ 159.716789] niks3-server[1169]: 2026/09/22 11:34:14 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo6234lb # [ 159.741258] niks3-server[1169]: 2026/09/22 11:34:14 INFO Completed upload id=516235worker1 # [ 159.786767] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.382Z level=INFO msg="Upload complete. (357ms)"6236worker1 # [ 159.799046] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48346 duration_s=0 paths=1 nar_bytes_in=4246237worker1 # [ 159.816774] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483466238worker1 # [ 159.826794] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6239worker1 # [ 159.835664] nix-daemon[852]: reaped child process 1697, status = succeeded6240client # [ 159.865923] nix[1375]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6241worker1 # [ 159.846015] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6242lb: must succeed: curl -sf localhost:9901/clusters6243lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6244lb: must succeed: curl -sf localhost:9901/clusters6245lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6246worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6247worker2 # [ 162.046059] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1476 (nix-grpc-daemon) on client request.6248worker2 # [ 162.063707] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6249worker1 # [ 162.077423] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6250lb # [ 162.029399] niks3-server[1169]: 2026/09/22 11:34:16 INFO lead: released remote=192.168.1.4:358726251client # [ 162.101481] nix[1375]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6252worker2 # [ 162.092602] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6253worker2 # [ 162.094878] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6254worker2 # [ 162.112733] systemd[1]: nix-grpc-daemon.service: Consumed 221ms CPU time over 5.991s wall clock time, 14.3M memory peak, 9.3K incoming IP traffic, 14.8K outgoing IP traffic.6255worker2 # [ 162.122951] systemd[1]: Starting scheduler-token.service...6256worker2 # [ 162.124320] nix-daemon[842]: reaped child process 1535, status = succeeded6257worker2 # [ 162.175443] systemd[1]: scheduler-token.service: Deactivated successfully.6258worker2 # [ 162.179822] systemd[1]: Finished scheduler-token.service.6259worker2 # [ 162.183947] systemd[1]: scheduler-token.service: Consumed 30ms CPU time over 64ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6260worker2 # [ 162.190059] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6261worker2 # [ 162.192509] systemd[1]: Starting scheduler-token.service...6262worker2 # [ 162.244213] systemd[1]: scheduler-token.service: Deactivated successfully.6263worker2 # [ 162.248268] systemd[1]: Finished scheduler-token.service.6264worker2 # [ 162.249366] systemd[1]: scheduler-token.service: Consumed 28ms CPU time over 57ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6265worker2 # [ 162.254745] systemd[1]: Starting Nix worker-protocol over gRPC...6266worker1 # [ 162.277568] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:16Z level=info event=scheduler_connected addr=lb:500516267worker2 # [ 162.359570] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6268worker2 # [ 162.361907] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6269worker2 # [ 162.365599] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16270worker2 # [ 162.370991] nix-grpc-daemon[1569]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6271lb # [ 162.334723] niks3-server[1169]: 2026/09/22 11:34:16 INFO lead: acquired remote=192.168.1.4:565366272worker2 # [ 162.388085] nix-grpc-daemon[1569]: E0000 00:00:1790076856.957215 1569 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6273worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.40 seconds)6274client: must succeed: sleep 5.06275worker2 # [ 162.416157] nix-grpc-daemon[1569]: E0000 00:00:1790076856.960736 1569 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6276worker2 # [ 162.418419] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6277worker2 # [ 162.424077] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=scheduler_take_over6278worker2 # [ 162.425637] nix-grpc-daemon[1569]: E0000 00:00:1790076856.971383 1608 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6279worker2 # [ 162.428173] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:350846280worker2 # [ 162.429698] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=16281worker2 # [ 162.434299] nix-grpc-daemon[1569]: E0000 00:00:1790076856.990836 1608 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6282worker2 # [ 162.439018] nix-grpc-daemon[1569]: E0000 00:00:1790076856.991023 1608 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6283worker2 # [ 162.441956] systemd[1]: Started Nix worker-protocol over gRPC.6284worker2 # [ 162.443247] nix-grpc-daemon[1569]: E0000 00:00:1790076857.010316 1608 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6285worker2 # [ 162.446021] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:17Z level=info event=scheduler_connected addr=lb:500516286worker2 # [ 162.449119] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:17Z level=info event=oidc_jwks provider=mock keys=16287worker2 # [ 162.450380] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:350846288worker2 # [ 162.452057] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06289client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6290worker1 # [ 167.676322] nix-daemon[852]: reaped child process 1727, status = succeeded6291client: waiting for success: ! systemctl is-active mid6292worker1 # [ 167.771304] nix-daemon[852]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6293worker1 # [ 167.787338] nix-daemon[852]: reaped child process 1767, status = succeeded6294lb # [ 167.749221] niks3-server[1169]: 2026/09/22 11:34:22 INFO Received uploads request method=POST path=/api/pending_closures6295worker1 # [ 167.807904] nix-grpc-daemon[1702]: time=2026-09-22T11:34:22.401Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6296worker1 # [ 167.810478] nix-grpc-daemon[1702]: time=2026-09-22T11:34:22.401Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6297lb # [ 167.786685] niks3-server[1169]: 2026/09/22 11:34:22 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6298lb # [ 167.818633] niks3-server[1169]: 2026/09/22 11:34:22 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6299lb # [ 167.832197] niks3-server[1169]: 2026/09/22 11:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6300lb # [ 167.839752] niks3-server[1169]: 2026/09/22 11:34:22 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6301lb # [ 167.846958] niks3-server[1169]: 2026/09/22 11:34:22 INFO Signed narinfos id=52 count=16302worker1 # [ 167.893790] nix-grpc-daemon[1702]: time=2026-09-22T11:34:22.487Z level=INFO msg="Uploading 1 narinfos"6303lb # [ 167.858348] niks3-server[1169]: 2026/09/22 11:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6304lb # [ 167.869665] niks3-server[1169]: 2026/09/22 11:34:22 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6305lb # [ 167.880423] niks3-server[1169]: 2026/09/22 11:34:22 INFO Completed upload id=526306worker1 # [ 167.925736] nix-grpc-daemon[1702]: time=2026-09-22T11:34:22.521Z level=INFO msg="Upload complete. (263ms)"6307worker1 # [ 167.940181] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48346 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16308worker1 # [ 167.957930] nix-daemon[852]: reaped child process 1725, status = succeeded6309client # [ 167.995733] systemd[1]: mid.service: Deactivated successfully.6310client # [ 167.999888] systemd[1]: mid.service: Consumed 206ms CPU time over 9.911s wall clock time, 39.5M memory peak, 8K incoming IP traffic, 8.8K outgoing IP traffic.6311lb # [ 168.336632] envoy[893]: conn peer=[2001:db8:1::4]:33604 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4102 tx=385 ms=58096312lb # [ 168.342187] envoy[893]: conn peer=[2001:db8:1::1]:53770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3078 tx=1629 ms=96746313client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6314client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6315client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)6316(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.87 seconds)6317subtest: interrupting the client stops the build on the worker6318client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6319client # [ 168.928848] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6320client # Running as unit: intr.service; invocation ID: b1299f6e02d54b6e80bd52a70e2d707d6321client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6322worker2 # [ 169.086923] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351006323worker2 # [ 169.099202] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6324client # [ 169.134143] nix[1411]: don't know how to build these paths:6325client # [ 169.135401] nix[1411]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6326worker2 # [ 169.125353] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:23Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06327worker2 # [ 169.130062] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:23Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16328worker2 # [ 169.173181] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351006329worker2 # [ 169.188606] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)6330worker2 # [ 169.276848] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 2/5)6331worker2 # [ 169.327484] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 160 ms (attempt 3/5)6332worker2 # [ 169.496776] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 549 ms (attempt 4/5)6333worker2 # [ 170.048302] nix-grpc-daemon[1569]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6334worker2 # [ 170.054582] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6335worker2 # [ 170.073043] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351006336client # [ 170.108225] nix[1411]: copying 1 paths to 'lb:50051'...6337client # [ 170.108535] nix[1411]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6338worker2 # [ 170.091544] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351006339worker2 # [ 170.105755] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6340worker2 # [ 170.133654] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:24Z level=info event=niks3_push_started pid=16436341worker2 # [ 170.300676] nix-grpc-daemon[1643]: time=2026-09-22T11:34:24.885Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6342worker2 # [ 170.407671] nix-daemon[842]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6343worker2 # [ 170.422340] nix-daemon[842]: reaped child process 1663, status = succeeded6344worker2 # [ 170.523193] nix-daemon[842]: accepted connection from pid 1665, user nix-grpc-daemon (trusted)6345worker2 # [ 170.538039] nix-daemon[842]: reaped child process 1668, status = succeeded6346lb # [ 170.497258] niks3-server[1169]: 2026/09/22 11:34:25 INFO Received uploads request method=POST path=/api/pending_closures6347worker2 # [ 170.560068] nix-grpc-daemon[1643]: time=2026-09-22T11:34:25.146Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6348worker2 # [ 170.561550] nix-grpc-daemon[1643]: time=2026-09-22T11:34:25.146Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6349lb # [ 170.541861] niks3-server[1169]: 2026/09/22 11:34:25 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6350worker2 # [ 170.603844] nix-grpc-daemon[1643]: time=2026-09-22T11:34:25.190Z level=INFO msg="Uploading 1 narinfos"6351lb # [ 170.556980] niks3-server[1169]: 2026/09/22 11:34:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6352lb # [ 170.566784] niks3-server[1169]: 2026/09/22 11:34:25 INFO Signed narinfos id=53 count=16353lb # [ 170.573446] niks3-server[1169]: 2026/09/22 11:34:25 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6354lb # [ 170.584271] niks3-server[1169]: 2026/09/22 11:34:25 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6355lb # [ 170.590109] niks3-server[1169]: 2026/09/22 11:34:25 INFO Completed upload id=536356worker2 # [ 170.638291] nix-grpc-daemon[1643]: time=2026-09-22T11:34:25.225Z level=INFO msg="Upload complete. (339ms)"6357lb # [ 170.592914] niks3-server[1169]: 2026/09/22 11:34:25 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6358worker2 # [ 170.649286] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=4246359worker2 # [ 170.662911] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351006360worker2 # [ 170.671979] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6361worker2 # [ 170.674464] nix-daemon[842]: reaped child process 1641, status = succeeded6362worker2 # [ 170.686505] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6363client # [ 170.705170] nix[1411]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6364client: must succeed: systemctl kill -s INT intr6365client # [ 172.645372] systemd[1]: intr.service: Sent signal SIGINT to main process 1411 (nix) on client request.6366worker2 # [ 172.636245] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:27Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6367client # [ 172.655406] nix[1411]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6368client # [ 172.665590] nix[1411]: error: interrupted by the user6369client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)6370client # [ 172.687817] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6371client # [ 172.692459] systemd[1]: intr.service: Failed with result 'exit-code'.6372client # [ 172.694313] systemd[1]: intr.service: Consumed 164ms CPU time over 3.761s wall clock time, 41.5M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6373worker2 # [ 172.714325] nix-daemon[842]: reaped child process 1670, status = succeeded6374worker2 # [ 172.716245] nix-daemon[1672]: unexpected Nix daemon error: error: interrupted by the user6375worker2 # [ 172.724527] nix-daemon[842]: reaped child process 1672, status = failed with exit code 16376(finished: subtest: interrupting the client stops the build on the worker, in 3.88 seconds)6377subtest: a deploy mid-build drains: the build finishes, the new generation takes over6378client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6379client # [ 172.833112] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6380client # Running as unit: sw.service; invocation ID: 395ecb4ffa5a47fc8823dd4a094cb39b6381client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)6382worker2 # [ 173.006756] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351006383client # [ 173.033153] nix[1432]: don't know how to build these paths:6384client # [ 173.034645] nix[1432]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6385worker2 # [ 173.024243] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:27Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06386worker2 # [ 173.031670] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:27Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16387worker1 # [ 173.075435] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483466388worker1 # [ 173.092364] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483466389client # [ 173.128515] nix[1432]: copying 1 paths to 'lb:50051'...6390worker1 # [ 173.107149] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483466391client # [ 173.132867] nix[1432]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6392worker1 # [ 173.122872] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6393worker1 # [ 173.291384] nix-daemon[852]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6394worker1 # [ 173.311037] nix-daemon[852]: reaped child process 1798, status = succeeded6395worker1 # [ 173.408114] nix-daemon[852]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6396worker1 # [ 173.423368] nix-daemon[852]: reaped child process 1803, status = succeeded6397lb # [ 173.386174] niks3-server[1169]: 2026/09/22 11:34:27 INFO Received uploads request method=POST path=/api/pending_closures6398worker1 # [ 173.443229] nix-grpc-daemon[1702]: time=2026-09-22T11:34:28.037Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6399worker1 # [ 173.444906] nix-grpc-daemon[1702]: time=2026-09-22T11:34:28.037Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6400lb # [ 173.416953] niks3-server[1169]: 2026/09/22 11:34:28 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6401lb # [ 173.431988] niks3-server[1169]: 2026/09/22 11:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6402worker1 # [ 173.482803] nix-grpc-daemon[1702]: time=2026-09-22T11:34:28.078Z level=INFO msg="Uploading 1 narinfos"6403lb # [ 173.443658] niks3-server[1169]: 2026/09/22 11:34:28 INFO Signed narinfos id=54 count=16404lb # [ 173.451242] niks3-server[1169]: 2026/09/22 11:34:28 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6405lb # [ 173.465949] niks3-server[1169]: 2026/09/22 11:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6406lb # [ 173.472923] niks3-server[1169]: 2026/09/22 11:34:28 INFO Completed upload id=546407worker1 # [ 173.517154] nix-grpc-daemon[1702]: time=2026-09-22T11:34:28.112Z level=INFO msg="Upload complete. (377ms)"6408lb # [ 173.476056] niks3-server[1169]: 2026/09/22 11:34:28 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6409worker1 # [ 173.524257] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48346 duration_s=0 paths=1 nar_bytes_in=4166410worker1 # [ 173.539248] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483466411worker1 # [ 173.547016] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6412worker1 # [ 173.552300] nix-daemon[852]: reaped child process 1793, status = succeeded6413client # [ 173.585970] nix[1432]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6414worker1 # [ 173.566615] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6415worker1: must succeed: systemctl reload nix-grpc-daemon.service6416worker1 # [ 175.415336] systemd[1]: Reloading Nix worker-protocol over gRPC...6417worker1 # [ 175.446137] systemd[1]: Reloaded Nix worker-protocol over gRPC.6418worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.07 seconds)6419worker1: must succeed: pgrep -f 'read -t [9]0 x'6420worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6421client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26422worker2 # [ 175.733689] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351006423client # don't know how to build these paths:6424client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6425worker2 # [ 175.742768] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351006426worker1 # [ 175.766178] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:30Z level=info event=draining6427worker2 # [ 175.771035] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06428worker2 # [ 175.777521] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16429worker2 # [ 175.816808] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351006430worker2 # [ 175.836757] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351006431client # copying 1 paths to 'lb:50051'...6432worker2 # [ 175.853221] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351006433client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6434worker2 # [ 175.876184] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6435worker2 # [ 176.042548] nix-daemon[842]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6436worker2 # [ 176.067878] nix-daemon[842]: reaped child process 1726, status = succeeded6437worker2 # [ 176.159018] nix-daemon[842]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)6438worker2 # [ 176.175223] nix-daemon[842]: reaped child process 1731, status = succeeded6439lb # [ 176.133188] niks3-server[1169]: 2026/09/22 11:34:30 INFO Received uploads request method=POST path=/api/pending_closures6440worker2 # [ 176.191744] nix-grpc-daemon[1643]: time=2026-09-22T11:34:30.778Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6441worker2 # [ 176.193440] nix-grpc-daemon[1643]: time=2026-09-22T11:34:30.778Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6442lb # [ 176.172294] niks3-server[1169]: 2026/09/22 11:34:30 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6443lb # [ 176.190191] niks3-server[1169]: 2026/09/22 11:34:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6444worker2 # [ 176.244144] nix-grpc-daemon[1643]: time=2026-09-22T11:34:30.831Z level=INFO msg="Uploading 1 narinfos"6445lb # [ 176.200172] niks3-server[1169]: 2026/09/22 11:34:30 INFO Signed narinfos id=55 count=16446lb # [ 176.205531] niks3-server[1169]: 2026/09/22 11:34:30 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6447lb # [ 176.214240] niks3-server[1169]: 2026/09/22 11:34:30 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6448worker2 # [ 176.270338] nix-grpc-daemon[1643]: time=2026-09-22T11:34:30.857Z level=INFO msg="Upload complete. (377ms)"6449lb # [ 176.223710] niks3-server[1169]: 2026/09/22 11:34:30 INFO Completed upload id=556450lb # [ 176.230621] niks3-server[1169]: 2026/09/22 11:34:30 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6451worker2 # [ 176.280384] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=5286452worker2 # [ 176.300447] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351006453worker2 # [ 176.305378] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6454worker2 # [ 176.316388] nix-daemon[842]: reaped child process 1721, status = succeeded6455client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6456client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6457worker2 # [ 176.325560] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6458client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6459client # farm-a> LOG-farm-a-during-drain6460worker2 # [ 177.596807] nix-daemon[842]: reaped child process 1735, status = succeeded6461worker2 # [ 177.703911] nix-daemon[842]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)6462worker2 # [ 177.724434] nix-daemon[842]: reaped child process 1754, status = succeeded6463lb # [ 177.674769] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received uploads request method=POST path=/api/pending_closures6464worker2 # [ 177.736514] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6465worker2 # [ 177.738390] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.321Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6466lb # [ 177.715258] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6467lb # [ 177.733707] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6468worker2 # [ 177.796122] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.382Z level=INFO msg="Uploading 1 narinfos"6469lb # [ 177.746027] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6470lb # [ 177.753674] niks3-server[1169]: 2026/09/22 11:34:32 INFO Signed narinfos id=56 count=16471lb # [ 177.755717] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6472lb # [ 177.763684] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6473lb # [ 177.771895] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6474worker2 # [ 177.820739] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.407Z level=INFO msg="Upload complete. (283ms)"6475lb # [ 177.776351] niks3-server[1169]: 2026/09/22 11:34:32 INFO Completed upload id=566476worker2 # [ 177.837928] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=16477worker2 # [ 177.855111] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06478worker2 # [ 177.859070] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16479worker2 # [ 177.863228] nix-daemon[842]: reaped child process 1733, status = succeeded6480worker2 # [ 177.882813] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351006481worker2 # [ 177.894091] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351006482worker2 # [ 177.908345] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.495Z level=INFO msg="All 1 paths already cached"6483client # copying 1 paths to 'lb:50051'...6484worker2 # [ 177.918922] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351006485client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6486worker2 # [ 177.941972] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6487worker2 # [ 178.101889] nix-daemon[842]: accepted connection from pid 1760, user nix-grpc-daemon (trusted)6488worker2 # [ 178.124305] nix-daemon[842]: reaped child process 1763, status = succeeded6489worker2 # [ 178.218053] nix-daemon[842]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6490worker2 # [ 178.235773] nix-daemon[842]: reaped child process 1768, status = succeeded6491lb # [ 178.185913] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received uploads request method=POST path=/api/pending_closures6492worker2 # [ 178.244160] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.831Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6493worker2 # [ 178.245654] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.831Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6494lb # [ 178.220933] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6495worker2 # [ 178.277017] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.863Z level=INFO msg="Uploading 1 narinfos"6496lb # [ 178.236918] niks3-server[1169]: 2026/09/22 11:34:32 INFO Signed narinfos id=57 count=16497lb # [ 178.245893] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6498lb # [ 178.259699] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6499worker2 # [ 178.314278] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.901Z level=INFO msg="Upload complete. (355ms)"6500lb # [ 178.268850] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6501worker2 # [ 178.320250] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=6406502lb # [ 178.274164] niks3-server[1169]: 2026/09/22 11:34:32 INFO Completed upload id=576503lb # [ 178.278248] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6504worker2 # [ 178.332945] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351006505worker2 # [ 178.338433] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6506worker2 # [ 178.343204] nix-daemon[842]: reaped child process 1758, status = succeeded6507worker2 # [ 178.353282] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6508client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6509client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6510lb # [ 178.330902] envoy[893]: conn peer=[2001:db8:1::1]:54320 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=829 ms=35756511client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6512client # farm-b> LOG-farm-b-during-drain6513worker2 # [ 179.858158] nix-daemon[842]: reaped child process 1772, status = succeeded6514worker2 # [ 179.933800] nix-daemon[842]: accepted connection from pid 1788, user nix-grpc-daemon (trusted)6515worker2 # [ 179.953546] nix-daemon[842]: reaped child process 1791, status = succeeded6516lb # [ 179.909221] niks3-server[1169]: 2026/09/22 11:34:34 INFO Received uploads request method=POST path=/api/pending_closures6517worker2 # [ 179.968537] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.554Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6518worker2 # [ 179.970302] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.554Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6519lb # [ 179.946721] niks3-server[1169]: 2026/09/22 11:34:34 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6520lb # [ 179.961122] niks3-server[1169]: 2026/09/22 11:34:34 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6521lb # [ 179.972913] niks3-server[1169]: 2026/09/22 11:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6522worker2 # [ 180.028108] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.614Z level=INFO msg="Uploading 1 narinfos"6523lb # [ 179.982034] niks3-server[1169]: 2026/09/22 11:34:34 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6524lb # [ 179.990904] niks3-server[1169]: 2026/09/22 11:34:34 INFO Signed narinfos id=58 count=16525lb # [ 180.001522] niks3-server[1169]: 2026/09/22 11:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6526lb # [ 180.008850] niks3-server[1169]: 2026/09/22 11:34:34 INFO Completed upload id=586527worker2 # [ 180.057074] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.644Z level=INFO msg="Upload complete. (292ms)"6528lb # [ 180.014097] niks3-server[1169]: 2026/09/22 11:34:34 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6529worker2 # [ 180.070270] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=16530worker2 # [ 180.094613] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06531worker2 # [ 180.099281] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16532worker2 # [ 180.104169] nix-daemon[842]: reaped child process 1770, status = succeeded6533worker2 # [ 180.133574] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351006534worker2 # [ 180.148787] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351006535worker2 # [ 180.162647] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.749Z level=INFO msg="All 2 paths already cached"6536client # copying 1 paths to 'lb:50051'...6537worker2 # [ 180.173524] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351006538client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6539worker2 # [ 180.196417] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6540worker2 # [ 180.353533] nix-daemon[842]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6541worker2 # [ 180.369824] nix-daemon[842]: reaped child process 1800, status = succeeded6542worker2 # [ 180.470225] nix-daemon[842]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6543worker2 # [ 180.489110] nix-daemon[842]: reaped child process 1805, status = succeeded6544lb # [ 180.438316] niks3-server[1169]: 2026/09/22 11:34:35 INFO Received uploads request method=POST path=/api/pending_closures6545worker2 # [ 180.495967] nix-grpc-daemon[1643]: time=2026-09-22T11:34:35.082Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6546worker2 # [ 180.497716] nix-grpc-daemon[1643]: time=2026-09-22T11:34:35.082Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6547lb # [ 180.463966] niks3-server[1169]: 2026/09/22 11:34:35 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6548lb # [ 180.480582] niks3-server[1169]: 2026/09/22 11:34:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6549worker2 # [ 180.535278] nix-grpc-daemon[1643]: time=2026-09-22T11:34:35.122Z level=INFO msg="Uploading 1 narinfos"6550lb # [ 180.493210] niks3-server[1169]: 2026/09/22 11:34:35 INFO Signed narinfos id=59 count=16551lb # [ 180.499995] niks3-server[1169]: 2026/09/22 11:34:35 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6552worker2 # [ 180.562598] nix-grpc-daemon[1643]: time=2026-09-22T11:34:35.149Z level=INFO msg="Upload complete. (349ms)"6553lb # [ 180.514404] niks3-server[1169]: 2026/09/22 11:34:35 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6554lb # [ 180.521321] niks3-server[1169]: 2026/09/22 11:34:35 INFO Completed upload id=596555worker2 # [ 180.569431] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=7686556lb # [ 180.523799] niks3-server[1169]: 2026/09/22 11:34:35 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6557worker2 # [ 180.581145] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351006558worker2 # [ 180.589531] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6559worker2 # [ 180.592398] nix-daemon[842]: reaped child process 1795, status = succeeded6560client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6561worker2 # [ 180.605273] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)6562client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6563client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6564client # farm-top> LOG-farm-top-during-drain6565worker2 # [ 181.466885] nix-daemon[842]: reaped child process 1809, status = succeeded6566worker2 # [ 181.580465] nix-daemon[842]: accepted connection from pid 1825, user nix-grpc-daemon (trusted)6567worker2 # [ 181.598859] nix-daemon[842]: reaped child process 1828, status = succeeded6568lb # [ 181.555267] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received uploads request method=POST path=/api/pending_closures6569worker2 # [ 181.615907] nix-grpc-daemon[1643]: time=2026-09-22T11:34:36.201Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6570worker2 # [ 181.618782] nix-grpc-daemon[1643]: time=2026-09-22T11:34:36.201Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6571lb # [ 181.595240] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6572worker2 # [ 181.666430] nix-grpc-daemon[1643]: time=2026-09-22T11:34:36.253Z level=INFO msg="Uploading 1 narinfos"6573lb # [ 181.610263] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6574lb # [ 181.622178] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6575lb # [ 181.629003] niks3-server[1169]: 2026/09/22 11:34:36 INFO Signed narinfos id=60 count=16576lb # [ 181.630987] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6577lb # [ 181.637286] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6578lb # [ 181.644292] niks3-server[1169]: 2026/09/22 11:34:36 INFO Completed upload id=606579worker2 # [ 181.693511] nix-grpc-daemon[1643]: time=2026-09-22T11:34:36.280Z level=INFO msg="Upload complete. (278ms)"6580lb # [ 181.646677] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6581worker2 # [ 181.709132] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=16582worker2 # [ 181.725389] nix-daemon[842]: reaped child process 1807, status = succeeded6583client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.26 seconds)6584client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6585client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6586worker1: must succeed: pkill -f 'read -t [9]0 x'6587worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6588client: waiting for success: ! systemctl is-active sw6589worker1 # [ 182.244982] nix-daemon[852]: reaped child process 1807, status = succeeded6590worker1 # [ 182.352841] nix-daemon[852]: accepted connection from pid 1848, user nix-grpc-daemon (trusted)6591worker1 # [ 182.374269] nix-daemon[852]: reaped child process 1851, status = succeeded6592lb # [ 182.327437] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received uploads request method=POST path=/api/pending_closures6593worker1 # [ 182.383235] nix-grpc-daemon[1702]: time=2026-09-22T11:34:36.977Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6594worker1 # [ 182.385568] nix-grpc-daemon[1702]: time=2026-09-22T11:34:36.977Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6595lb # [ 182.359032] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6596lb # [ 182.378501] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6597worker1 # [ 182.432723] nix-grpc-daemon[1702]: time=2026-09-22T11:34:37.027Z level=INFO msg="Uploading 1 narinfos"6598lb # [ 182.391652] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6599lb # [ 182.401039] niks3-server[1169]: 2026/09/22 11:34:36 INFO Signed narinfos id=61 count=16600lb # [ 182.410206] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6601worker1 # [ 182.455178] nix-grpc-daemon[1702]: time=2026-09-22T11:34:37.050Z level=INFO msg="Upload complete. (270ms)"6602lb # [ 182.422434] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6603lb # [ 182.430367] niks3-server[1169]: 2026/09/22 11:34:37 INFO Completed upload id=616604worker1 # [ 182.473165] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48346 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=16605lb # [ 182.433075] niks3-server[1169]: 2026/09/22 11:34:37 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6606worker1 # [ 182.489792] nix-daemon[852]: reaped child process 1805, status = succeeded6607client # [ 182.523653] systemd[1]: sw.service: Deactivated successfully.6608client # [ 182.528205] systemd[1]: sw.service: Consumed 187ms CPU time over 9.694s wall clock time, 43.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6609worker1 # [ 182.767952] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:37Z level=info event=signal_exit6610client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6611client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6612client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6613worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6614worker1 # [ 184.762237] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:39Z level=info event=scheduler_restarting6615worker2 # [ 184.777418] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:350846616worker2 # [ 184.782131] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:39Z level=info event=worker_gone addr=192.168.1.3:500516617worker1 # [ 184.794231] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6618worker1 # [ 184.819885] nix-daemon[852]: reaped child process 1691, status = succeeded6619worker1 # [ 185.465384] nix-grpc-daemon[1579]: E0000 00:00:1790076880.060842 1624 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6620worker1 # [ 185.474391] nix-grpc-daemon[1579]: E0000 00:00:1790076880.069892 1624 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6621worker1 # [ 186.158891] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6622worker1 # [ 186.163559] systemd[1]: nix-grpc-daemon.service: Consumed 1.154s CPU time over 42.563s wall clock time, 36.4M memory peak, 26.4K incoming IP traffic, 38.1K outgoing IP traffic.6623worker1 # [ 186.187783] systemd[1]: Starting Nix worker-protocol over gRPC...6624worker1 # [ 186.337998] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6625worker1 # [ 186.340279] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6626worker1 # [ 186.344149] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16627worker1 # [ 186.349424] nix-grpc-daemon[1865]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6628worker1 # [ 186.350901] nix-grpc-daemon[1865]: E0000 00:00:1790076880.944236 1865 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6629worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.24 seconds)6630lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6631lb: must succeed: curl -sf localhost:9901/clusters6632worker1 # [ 186.380174] nix-grpc-daemon[1865]: E0000 00:00:1790076880.950272 1865 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6633worker1 # [ 186.382501] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6634worker1 # [ 186.400475] nix-grpc-daemon[1865]: E0000 00:00:1790076880.960742 1905 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6635worker1 # [ 186.403369] systemd[1]: Started Nix worker-protocol over gRPC.6636worker1 # [ 186.404406] nix-grpc-daemon[1865]: E0000 00:00:1790076880.980617 1905 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6637worker1 # [ 186.406696] nix-grpc-daemon[1865]: E0000 00:00:1790076880.980777 1905 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6638worker1 # [ 186.409207] nix-grpc-daemon[1865]: E0000 00:00:1790076880.995160 1905 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6639worker1 # [ 186.417316] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:41Z level=info event=scheduler_connected addr=lb:500516640worker2 # [ 186.424625] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:41Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:350846641worker2 # [ 186.429923] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:41Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06642lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6643lb: must succeed: curl -sf localhost:9901/clusters6644lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6645lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)6646lb: must succeed: curl -sf localhost:9901/clusters6647lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6648worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6649worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6650(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.89 seconds)6651subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6652client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16653worker2 # [ 187.868370] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351006654worker2 # [ 187.874684] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351006655worker2 # [ 187.920331] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06656worker2 # [ 187.926733] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16657worker2 # [ 187.933006] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06658worker2 # [ 187.938289] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06659worker1 # [ 187.978373] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286660worker1 # [ 187.991907] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6661worker1 # [ 188.016148] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5)6662worker1 # [ 188.092807] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 2/5)6663worker1 # [ 188.102224] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 3/5)6664worker1 # [ 188.156726] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 548 ms (attempt 4/5)6665lb # [ 188.333535] envoy[893]: conn peer=[2001:db8:1::1]:51980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9664 tx=4396 ms=59936666lb # [ 188.339005] envoy[893]: conn peer=[2001:db8:1::1]:51972 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1517 ms=94826667lb # [ 188.343965] envoy[893]: conn peer=[2001:db8:1::3]:37470 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2297 tx=738 ms=409706668worker1 # [ 188.707417] nix-grpc-daemon[1865]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6669worker1 # [ 188.713864] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6670worker1 # [ 188.732375] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345286671worker1 # [ 188.748690] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345286672worker1 # [ 188.762742] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6673worker1 # [ 188.801915] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=info event=niks3_push_started pid=19256674worker1 # [ 188.955718] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.549Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6675worker1 # [ 189.059842] nix-daemon[852]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)6676worker1 # [ 189.077987] nix-daemon[852]: reaped child process 1941, status = succeeded6677worker1 # [ 189.177238] nix-daemon[852]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)6678worker1 # [ 189.194852] nix-daemon[852]: reaped child process 1946, status = succeeded6679lb # [ 189.155976] niks3-server[1169]: 2026/09/22 11:34:43 INFO Received uploads request method=POST path=/api/pending_closures6680worker1 # [ 189.211536] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.806Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6681worker1 # [ 189.213935] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.806Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6682lb # [ 189.193721] niks3-server[1169]: 2026/09/22 11:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6683worker1 # [ 189.248428] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.843Z level=INFO msg="Uploading 1 narinfos"6684lb # [ 189.206559] niks3-server[1169]: 2026/09/22 11:34:43 INFO Signed narinfos id=62 count=16685lb # [ 189.219540] niks3-server[1169]: 2026/09/22 11:34:43 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6686lb # [ 189.229259] niks3-server[1169]: 2026/09/22 11:34:43 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6687worker1 # [ 189.283349] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.878Z level=INFO msg="Upload complete. (328ms)"6688lb # [ 189.239651] niks3-server[1169]: 2026/09/22 11:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6689lb # [ 189.247661] niks3-server[1169]: 2026/09/22 11:34:43 INFO Completed upload id=626690worker1 # [ 189.293380] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=5126691lb # [ 189.252267] niks3-server[1169]: 2026/09/22 11:34:43 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6692worker1 # [ 189.306957] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286693worker1 # [ 189.312566] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6694worker1 # [ 189.316912] nix-daemon[852]: reaped child process 1923, status = succeeded6695worker1 # [ 189.328160] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6696worker1 # [ 190.839215] nix-daemon[852]: reaped child process 1950, status = succeeded6697worker1 # [ 190.910695] nix-daemon[852]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)6698worker1 # [ 190.931011] nix-daemon[852]: reaped child process 1969, status = succeeded6699lb # [ 190.885373] niks3-server[1169]: 2026/09/22 11:34:45 INFO Received uploads request method=POST path=/api/pending_closures6700worker1 # [ 190.940463] nix-grpc-daemon[1925]: time=2026-09-22T11:34:45.535Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6701worker1 # [ 190.941970] nix-grpc-daemon[1925]: time=2026-09-22T11:34:45.535Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6702lb # [ 190.917694] niks3-server[1169]: 2026/09/22 11:34:45 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6703lb # [ 190.940486] niks3-server[1169]: 2026/09/22 11:34:45 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6704worker1 # [ 191.004112] nix-grpc-daemon[1925]: time=2026-09-22T11:34:45.598Z level=INFO msg="Uploading 1 narinfos"6705lb # [ 190.952789] niks3-server[1169]: 2026/09/22 11:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6706lb # [ 190.964580] niks3-server[1169]: 2026/09/22 11:34:45 INFO Signed narinfos id=63 count=16707lb # [ 190.967234] niks3-server[1169]: 2026/09/22 11:34:45 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6708lb # [ 190.979990] niks3-server[1169]: 2026/09/22 11:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6709lb # [ 190.987366] niks3-server[1169]: 2026/09/22 11:34:45 INFO Completed upload id=636710worker1 # [ 191.031869] nix-grpc-daemon[1925]: time=2026-09-22T11:34:45.627Z level=INFO msg="Upload complete. (293ms)"6711lb # [ 190.990712] niks3-server[1169]: 2026/09/22 11:34:45 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6712worker2 # [ 191.046267] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:45Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16713worker1 # [ 191.045175] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=16714worker2 # [ 191.059859] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:45Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06715worker1 # [ 191.065302] nix-daemon[852]: reaped child process 1948, status = succeeded6716worker1 # [ 191.098274] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286717worker1 # [ 191.113327] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345286718worker1 # [ 191.127103] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345286719worker1 # [ 191.140111] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6720worker1 # [ 191.296330] nix-daemon[852]: accepted connection from pid 1973, user nix-grpc-daemon (trusted)6721worker1 # [ 191.312852] nix-daemon[852]: reaped child process 1976, status = succeeded6722worker1 # [ 191.414074] nix-daemon[852]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)6723worker1 # [ 191.431903] nix-daemon[852]: reaped child process 1981, status = succeeded6724lb # [ 191.385394] niks3-server[1169]: 2026/09/22 11:34:45 INFO Received uploads request method=POST path=/api/pending_closures6725worker1 # [ 191.438778] nix-grpc-daemon[1925]: time=2026-09-22T11:34:46.034Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6726worker1 # [ 191.440421] nix-grpc-daemon[1925]: time=2026-09-22T11:34:46.034Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6727lb # [ 191.412971] niks3-server[1169]: 2026/09/22 11:34:46 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6728worker1 # [ 191.471423] nix-grpc-daemon[1925]: time=2026-09-22T11:34:46.066Z level=INFO msg="Uploading 1 narinfos"6729lb # [ 191.426384] niks3-server[1169]: 2026/09/22 11:34:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6730lb # [ 191.440468] niks3-server[1169]: 2026/09/22 11:34:46 INFO Signed narinfos id=64 count=16731lb # [ 191.443747] niks3-server[1169]: 2026/09/22 11:34:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6732worker1 # [ 191.505701] nix-grpc-daemon[1925]: time=2026-09-22T11:34:46.101Z level=INFO msg="Upload complete. (350ms)"6733lb # [ 191.461121] niks3-server[1169]: 2026/09/22 11:34:46 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6734lb # [ 191.466988] niks3-server[1169]: 2026/09/22 11:34:46 INFO Completed upload id=646735lb # [ 191.470514] niks3-server[1169]: 2026/09/22 11:34:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6736worker1 # [ 191.515832] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=5126737worker1 # [ 191.526838] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286738worker1 # [ 191.534609] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6739worker1 # [ 191.536894] nix-daemon[852]: reaped child process 1971, status = succeeded6740worker1 # [ 191.548535] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6741worker1 # [ 192.629208] nix-daemon[852]: reaped child process 1985, status = succeeded6742worker1 # [ 192.695391] nix-daemon[852]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)6743worker1 # [ 192.713815] nix-daemon[852]: reaped child process 2004, status = succeeded6744lb # [ 192.675108] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received uploads request method=POST path=/api/pending_closures6745worker1 # [ 192.730061] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.324Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6746worker1 # [ 192.731513] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.324Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6747lb # [ 192.713397] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6748lb # [ 192.725524] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6749worker1 # [ 192.782704] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.377Z level=INFO msg="Uploading 1 narinfos"6750lb # [ 192.740566] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6751lb # [ 192.749221] niks3-server[1169]: 2026/09/22 11:34:47 INFO Signed narinfos id=65 count=16752lb # [ 192.751731] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6753lb # [ 192.764265] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6754lb # [ 192.770548] niks3-server[1169]: 2026/09/22 11:34:47 INFO Completed upload id=656755worker1 # [ 192.813763] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.409Z level=INFO msg="Upload complete. (256ms)"6756lb # [ 192.772906] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6757worker2 # [ 192.831946] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:47Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16758worker1 # [ 192.828005] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=16759worker2 # [ 192.840070] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:47Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06760worker1 # [ 192.847446] nix-daemon[852]: reaped child process 1983, status = succeeded6761worker1 # [ 192.874009] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286762worker1 # [ 192.889460] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345286763worker1 # [ 192.904742] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345286764worker1 # [ 192.918490] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6765worker1 # [ 193.073552] nix-daemon[852]: accepted connection from pid 2008, user nix-grpc-daemon (trusted)6766worker1 # [ 193.090444] nix-daemon[852]: reaped child process 2011, status = succeeded6767worker1 # [ 193.195400] nix-daemon[852]: accepted connection from pid 2013, user nix-grpc-daemon (trusted)6768worker1 # [ 193.212615] nix-daemon[852]: reaped child process 2016, status = succeeded6769lb # [ 193.172802] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received uploads request method=POST path=/api/pending_closures6770worker1 # [ 193.226142] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.821Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6771worker1 # [ 193.227724] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.821Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6772worker1 # [ 193.262202] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.857Z level=INFO msg="Uploading 1 narinfos"6773lb # [ 193.205533] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6774lb # [ 193.219941] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6775lb # [ 193.230307] niks3-server[1169]: 2026/09/22 11:34:47 INFO Signed narinfos id=66 count=16776lb # [ 193.239786] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6777lb # [ 193.250352] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6778lb # [ 193.255860] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6779lb # [ 193.261470] niks3-server[1169]: 2026/09/22 11:34:47 INFO Completed upload id=666780worker1 # [ 193.305537] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.901Z level=INFO msg="Upload complete. (371ms)"6781worker1 # [ 193.314904] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=5126782worker1 # [ 193.330798] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286783worker1 # [ 193.337590] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6784worker1 # [ 193.342927] nix-daemon[852]: reaped child process 2006, status = succeeded6785worker1 # [ 193.354947] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6786worker1 # [ 194.036346] nix-daemon[852]: reaped child process 2020, status = succeeded6787worker1 # [ 194.137359] nix-daemon[852]: accepted connection from pid 2036, user nix-grpc-daemon (trusted)6788worker1 # [ 194.154136] nix-daemon[852]: reaped child process 2039, status = succeeded6789lb # [ 194.117043] niks3-server[1169]: 2026/09/22 11:34:48 INFO Received uploads request method=POST path=/api/pending_closures6790worker1 # [ 194.172430] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.766Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6791worker1 # [ 194.174181] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.766Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6792lb # [ 194.147963] niks3-server[1169]: 2026/09/22 11:34:48 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6793lb # [ 194.162029] niks3-server[1169]: 2026/09/22 11:34:48 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6794worker1 # [ 194.225149] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.820Z level=INFO msg="Uploading 1 narinfos"6795lb # [ 194.179660] niks3-server[1169]: 2026/09/22 11:34:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6796lb # [ 194.190696] niks3-server[1169]: 2026/09/22 11:34:48 INFO Signed narinfos id=67 count=16797lb # [ 194.197649] niks3-server[1169]: 2026/09/22 11:34:48 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6798worker1 # [ 194.255374] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.850Z level=INFO msg="Upload complete. (286ms)"6799lb # [ 194.209694] niks3-server[1169]: 2026/09/22 11:34:48 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6800lb # [ 194.218856] niks3-server[1169]: 2026/09/22 11:34:48 INFO Completed upload id=676801lb # [ 194.222614] niks3-server[1169]: 2026/09/22 11:34:48 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6802worker2 # [ 194.269120] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:48Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16803worker1 # [ 194.272243] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=16804worker2 # [ 194.282684] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:48Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06805worker1 # [ 194.288665] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286806worker1 # [ 194.297306] nix-daemon[852]: reaped child process 2018, status = succeeded6807worker1 # [ 194.305567] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345286808worker1 # [ 194.319515] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.915Z level=INFO msg="All 1 paths already cached"6809worker1 # [ 194.329276] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345286810worker1 # [ 194.342317] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6811worker1 # [ 194.495002] nix-daemon[852]: accepted connection from pid 2043, user nix-grpc-daemon (trusted)6812worker1 # [ 194.515587] nix-daemon[852]: reaped child process 2046, status = succeeded6813worker1 # [ 194.613845] nix-daemon[852]: accepted connection from pid 2048, user nix-grpc-daemon (trusted)6814worker1 # [ 194.631008] nix-daemon[852]: reaped child process 2051, status = succeeded6815lb # [ 194.589532] niks3-server[1169]: 2026/09/22 11:34:49 INFO Received uploads request method=POST path=/api/pending_closures6816worker1 # [ 194.642867] nix-grpc-daemon[1925]: time=2026-09-22T11:34:49.238Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6817worker1 # [ 194.644391] nix-grpc-daemon[1925]: time=2026-09-22T11:34:49.238Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6818lb # [ 194.616591] niks3-server[1169]: 2026/09/22 11:34:49 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6819worker1 # [ 194.675065] nix-grpc-daemon[1925]: time=2026-09-22T11:34:49.270Z level=INFO msg="Uploading 1 narinfos"6820lb # [ 194.630877] niks3-server[1169]: 2026/09/22 11:34:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6821lb # [ 194.643408] niks3-server[1169]: 2026/09/22 11:34:49 INFO Signed narinfos id=68 count=16822lb # [ 194.648988] niks3-server[1169]: 2026/09/22 11:34:49 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6823lb # [ 194.658788] niks3-server[1169]: 2026/09/22 11:34:49 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6824worker1 # [ 194.710454] nix-grpc-daemon[1925]: time=2026-09-22T11:34:49.306Z level=INFO msg="Upload complete. (353ms)"6825lb # [ 194.665787] niks3-server[1169]: 2026/09/22 11:34:49 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6826lb # [ 194.668993] niks3-server[1169]: 2026/09/22 11:34:49 INFO Completed upload id=686827worker1 # [ 194.719750] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=6246828worker1 # [ 194.732223] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286829worker1 # [ 194.739186] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6830worker1 # [ 194.743855] nix-daemon[852]: reaped child process 2041, status = succeeded6831worker1 # [ 194.755668] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6832worker1 # [ 196.246575] nix-daemon[852]: reaped child process 2055, status = succeeded6833worker1 # [ 196.316444] nix-daemon[852]: accepted connection from pid 2071, user nix-grpc-daemon (trusted)6834worker1 # [ 196.333482] nix-daemon[852]: reaped child process 2074, status = succeeded6835lb # [ 196.298231] niks3-server[1169]: 2026/09/22 11:34:50 INFO Received uploads request method=POST path=/api/pending_closures6836worker1 # [ 196.356440] nix-grpc-daemon[1925]: time=2026-09-22T11:34:50.951Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6837worker1 # [ 196.358232] nix-grpc-daemon[1925]: time=2026-09-22T11:34:50.951Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6838lb # [ 196.348684] niks3-server[1169]: 2026/09/22 11:34:50 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6839lb # [ 196.357100] niks3-server[1169]: 2026/09/22 11:34:50 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6840worker1 # [ 196.409801] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.005Z level=INFO msg="Uploading 1 narinfos"6841lb # [ 196.365946] niks3-server[1169]: 2026/09/22 11:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6842lb # [ 196.372224] niks3-server[1169]: 2026/09/22 11:34:50 INFO Signed narinfos id=69 count=16843lb # [ 196.376945] niks3-server[1169]: 2026/09/22 11:34:50 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6844lb # [ 196.384740] niks3-server[1169]: 2026/09/22 11:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6845lb # [ 196.390320] niks3-server[1169]: 2026/09/22 11:34:50 INFO Completed upload id=696846lb # [ 196.393341] niks3-server[1169]: 2026/09/22 11:34:50 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6847worker1 # [ 196.433919] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.029Z level=INFO msg="Upload complete. (264ms)"6848worker2 # [ 196.447875] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:51Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16849worker1 # [ 196.445273] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=16850worker2 # [ 196.460298] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:51Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06851worker1 # [ 196.465391] nix-daemon[852]: reaped child process 2053, status = succeeded6852worker1 # [ 196.500542] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286853worker1 # [ 196.516885] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345286854worker1 # [ 196.531101] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.126Z level=INFO msg="All 1 paths already cached"6855worker1 # [ 196.540766] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345286856worker1 # [ 196.555945] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6857worker1 # [ 196.713188] nix-daemon[852]: accepted connection from pid 2078, user nix-grpc-daemon (trusted)6858worker1 # [ 196.737578] nix-daemon[852]: reaped child process 2081, status = succeeded6859worker1 # [ 196.830933] nix-daemon[852]: accepted connection from pid 2083, user nix-grpc-daemon (trusted)6860worker1 # [ 196.845432] nix-daemon[852]: reaped child process 2086, status = succeeded6861lb # [ 196.807398] niks3-server[1169]: 2026/09/22 11:34:51 INFO Received uploads request method=POST path=/api/pending_closures6862worker1 # [ 196.862087] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.457Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6863worker1 # [ 196.863545] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.457Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6864lb # [ 196.834846] niks3-server[1169]: 2026/09/22 11:34:51 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6865worker1 # [ 196.893452] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.488Z level=INFO msg="Uploading 1 narinfos"6866lb # [ 196.849830] niks3-server[1169]: 2026/09/22 11:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6867lb # [ 196.859963] niks3-server[1169]: 2026/09/22 11:34:51 INFO Signed narinfos id=70 count=16868lb # [ 196.867412] niks3-server[1169]: 2026/09/22 11:34:51 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6869lb # [ 196.877059] niks3-server[1169]: 2026/09/22 11:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6870worker1 # [ 196.928411] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.524Z level=INFO msg="Upload complete. (356ms)"6871lb # [ 196.886671] niks3-server[1169]: 2026/09/22 11:34:51 INFO Completed upload id=706872lb # [ 196.889876] niks3-server[1169]: 2026/09/22 11:34:51 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6873worker1 # [ 196.936279] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=6246874worker1 # [ 196.950638] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286875worker1 # [ 196.956898] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6876worker1 # [ 196.961885] nix-daemon[852]: reaped child process 2076, status = succeeded6877worker1 # [ 196.973738] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6878worker1 # [ 197.844260] nix-daemon[852]: reaped child process 2090, status = succeeded6879worker1 # [ 197.954939] nix-daemon[852]: accepted connection from pid 2106, user nix-grpc-daemon (trusted)6880worker1 # [ 197.978407] nix-daemon[852]: reaped child process 2109, status = succeeded6881lb # [ 197.938974] niks3-server[1169]: 2026/09/22 11:34:52 INFO Received uploads request method=POST path=/api/pending_closures6882worker1 # [ 197.996603] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.589Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6883worker1 # [ 197.998574] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.590Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6884lb # [ 197.987362] niks3-server[1169]: 2026/09/22 11:34:52 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6885lb # [ 197.999656] niks3-server[1169]: 2026/09/22 11:34:52 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6886worker1 # [ 198.056736] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.652Z level=INFO msg="Uploading 1 narinfos"6887lb # [ 198.011686] niks3-server[1169]: 2026/09/22 11:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6888lb # [ 198.020341] niks3-server[1169]: 2026/09/22 11:34:52 INFO Signed narinfos id=71 count=16889lb # [ 198.026371] niks3-server[1169]: 2026/09/22 11:34:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6890lb # [ 198.035713] niks3-server[1169]: 2026/09/22 11:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6891worker1 # [ 198.085001] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.680Z level=INFO msg="Upload complete. (301ms)"6892lb # [ 198.042074] niks3-server[1169]: 2026/09/22 11:34:52 INFO Completed upload id=716893lb # [ 198.044810] niks3-server[1169]: 2026/09/22 11:34:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6894worker2 # [ 198.099906] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:52Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16895worker1 # [ 198.099728] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=16896worker2 # [ 198.112934] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:52Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06897worker1 # [ 198.116297] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286898worker1 # [ 198.129870] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345286899worker1 # [ 198.136330] nix-daemon[852]: reaped child process 2088, status = succeeded6900worker1 # [ 198.143431] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.739Z level=INFO msg="All 1 paths already cached"6901worker1 # [ 198.153952] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345286902worker1 # [ 198.167324] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6903worker1 # [ 198.312228] nix-daemon[852]: accepted connection from pid 2113, user nix-grpc-daemon (trusted)6904worker1 # [ 198.333012] nix-daemon[852]: reaped child process 2116, status = succeeded6905worker1 # [ 198.436111] nix-daemon[852]: accepted connection from pid 2118, user nix-grpc-daemon (trusted)6906worker1 # [ 198.452813] nix-daemon[852]: reaped child process 2121, status = succeeded6907lb # [ 198.413416] niks3-server[1169]: 2026/09/22 11:34:53 INFO Received uploads request method=POST path=/api/pending_closures6908worker1 # [ 198.469727] nix-grpc-daemon[1925]: time=2026-09-22T11:34:53.065Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6909worker1 # [ 198.471199] nix-grpc-daemon[1925]: time=2026-09-22T11:34:53.065Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6910lb # [ 198.443111] niks3-server[1169]: 2026/09/22 11:34:53 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6911lb # [ 198.459256] niks3-server[1169]: 2026/09/22 11:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6912worker1 # [ 198.510325] nix-grpc-daemon[1925]: time=2026-09-22T11:34:53.105Z level=INFO msg="Uploading 1 narinfos"6913lb # [ 198.470735] niks3-server[1169]: 2026/09/22 11:34:53 INFO Signed narinfos id=72 count=16914lb # [ 198.476753] niks3-server[1169]: 2026/09/22 11:34:53 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6915lb # [ 198.486639] niks3-server[1169]: 2026/09/22 11:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6916lb # [ 198.499204] niks3-server[1169]: 2026/09/22 11:34:53 INFO Completed upload id=726917lb # [ 198.502130] niks3-server[1169]: 2026/09/22 11:34:53 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6918worker1 # [ 198.542641] nix-grpc-daemon[1925]: time=2026-09-22T11:34:53.138Z level=INFO msg="Upload complete. (362ms)"6919worker1 # [ 198.552434] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=6246920worker1 # [ 198.567148] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286921worker1 # [ 198.573980] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6922worker1 # [ 198.578919] nix-daemon[852]: reaped child process 2111, status = succeeded6923worker1 # [ 198.591530] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6924worker1 # [ 200.259751] nix-daemon[852]: reaped child process 2125, status = succeeded6925worker1 # [ 200.392483] nix-daemon[852]: accepted connection from pid 2141, user nix-grpc-daemon (trusted)6926worker1 # [ 200.407712] nix-daemon[852]: reaped child process 2144, status = succeeded6927lb # [ 200.370850] niks3-server[1169]: 2026/09/22 11:34:54 INFO Received uploads request method=POST path=/api/pending_closures6928worker1 # [ 200.427682] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.020Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6929worker1 # [ 200.429559] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.020Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6930lb # [ 200.407508] niks3-server[1169]: 2026/09/22 11:34:54 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6931lb # [ 200.425565] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6932lb # [ 200.436774] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6933worker1 # [ 200.486401] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.081Z level=INFO msg="Uploading 1 narinfos"6934lb # [ 200.444383] niks3-server[1169]: 2026/09/22 11:34:55 INFO Signed narinfos id=73 count=16935lb # [ 200.451837] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6936lb # [ 200.462222] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6937worker1 # [ 200.515322] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.110Z level=INFO msg="Upload complete. (311ms)"6938lb # [ 200.473681] niks3-server[1169]: 2026/09/22 11:34:55 INFO Completed upload id=736939lb # [ 200.478547] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6940worker2 # [ 200.528692] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:55Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16941worker1 # [ 200.527715] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=16942worker2 # [ 200.541820] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:55Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06943worker1 # [ 200.551397] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286944worker1 # [ 200.557346] nix-daemon[852]: reaped child process 2123, status = succeeded6945worker1 # [ 200.566150] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345286946worker1 # [ 200.580784] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.175Z level=INFO msg="All 2 paths already cached"6947worker1 # [ 200.590407] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345286948worker1 # [ 200.604114] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6949worker1 # [ 200.748911] nix-daemon[852]: accepted connection from pid 2148, user nix-grpc-daemon (trusted)6950worker1 # [ 200.774509] nix-daemon[852]: reaped child process 2151, status = succeeded6951worker1 # [ 200.868008] nix-daemon[852]: accepted connection from pid 2153, user nix-grpc-daemon (trusted)6952worker1 # [ 200.881745] nix-daemon[852]: reaped child process 2156, status = succeeded6953lb # [ 200.844349] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received uploads request method=POST path=/api/pending_closures6954worker1 # [ 200.899341] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.494Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6955worker1 # [ 200.900906] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.494Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6956lb # [ 200.872384] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6957lb # [ 200.886511] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6958worker1 # [ 200.938811] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.534Z level=INFO msg="Uploading 1 narinfos"6959lb # [ 200.900097] niks3-server[1169]: 2026/09/22 11:34:55 INFO Signed narinfos id=74 count=16960lb # [ 200.902771] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6961worker1 # [ 200.960356] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.556Z level=INFO msg="Upload complete. (341ms)"6962lb # [ 200.918038] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6963lb # [ 200.923444] niks3-server[1169]: 2026/09/22 11:34:55 INFO Completed upload id=746964worker1 # [ 200.968266] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=7526965lb # [ 200.926859] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6966worker1 # [ 200.979581] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286967worker1 # [ 200.988817] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6968worker1 # [ 200.992060] nix-daemon[852]: reaped child process 2146, status = succeeded6969worker1 # [ 201.002427] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6970worker1 # [ 201.860965] nix-daemon[852]: reaped child process 2160, status = succeeded6971worker1 # [ 201.997979] nix-daemon[852]: accepted connection from pid 2176, user nix-grpc-daemon (trusted)6972worker1 # [ 202.018772] nix-daemon[852]: reaped child process 2179, status = succeeded6973lb # [ 201.972895] niks3-server[1169]: 2026/09/22 11:34:56 INFO Received uploads request method=POST path=/api/pending_closures6974worker1 # [ 202.028466] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.622Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6975worker1 # [ 202.030141] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.622Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6976lb # [ 202.009399] niks3-server[1169]: 2026/09/22 11:34:56 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6977lb # [ 202.021909] niks3-server[1169]: 2026/09/22 11:34:56 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6978worker1 # [ 202.080053] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.675Z level=INFO msg="Uploading 1 narinfos"6979lb # [ 202.029811] niks3-server[1169]: 2026/09/22 11:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6980lb # [ 202.040259] niks3-server[1169]: 2026/09/22 11:34:56 INFO Signed narinfos id=75 count=16981lb # [ 202.046322] niks3-server[1169]: 2026/09/22 11:34:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6982lb # [ 202.057823] niks3-server[1169]: 2026/09/22 11:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6983lb # [ 202.065695] niks3-server[1169]: 2026/09/22 11:34:56 INFO Completed upload id=756984worker1 # [ 202.109682] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.705Z level=INFO msg="Upload complete. (294ms)"6985lb # [ 202.068415] niks3-server[1169]: 2026/09/22 11:34:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6986worker2 # [ 202.123853] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:56Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16987worker1 # [ 202.121187] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=16988worker1 # [ 202.142531] nix-daemon[852]: reaped child process 2158, status = succeeded6989worker1 # [ 202.172903] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345286990worker1 # [ 202.188472] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345286991worker1 # [ 202.201759] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.797Z level=INFO msg="All 2 paths already cached"6992worker1 # [ 202.213053] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345286993worker1 # [ 202.226682] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6994worker1 # [ 202.369882] nix-daemon[852]: accepted connection from pid 2183, user nix-grpc-daemon (trusted)6995worker1 # [ 202.386957] nix-daemon[852]: reaped child process 2186, status = succeeded6996worker1 # [ 202.487267] nix-daemon[852]: accepted connection from pid 2188, user nix-grpc-daemon (trusted)6997worker1 # [ 202.505270] nix-daemon[852]: reaped child process 2191, status = succeeded6998lb # [ 202.458166] niks3-server[1169]: 2026/09/22 11:34:57 INFO Received uploads request method=POST path=/api/pending_closures6999worker1 # [ 202.511500] nix-grpc-daemon[1925]: time=2026-09-22T11:34:57.106Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7000worker1 # [ 202.513195] nix-grpc-daemon[1925]: time=2026-09-22T11:34:57.106Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"7001lb # [ 202.484246] niks3-server[1169]: 2026/09/22 11:34:57 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst7002worker1 # [ 202.542944] nix-grpc-daemon[1925]: time=2026-09-22T11:34:57.138Z level=INFO msg="Uploading 1 narinfos"7003lb # [ 202.498644] niks3-server[1169]: 2026/09/22 11:34:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7004lb # [ 202.508387] niks3-server[1169]: 2026/09/22 11:34:57 INFO Signed narinfos id=76 count=17005worker1 # [ 202.570837] nix-grpc-daemon[1925]: time=2026-09-22T11:34:57.166Z level=INFO msg="Upload complete. (328ms)"7006lb # [ 202.517737] niks3-server[1169]: 2026/09/22 11:34:57 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7007lb # [ 202.533106] niks3-server[1169]: 2026/09/22 11:34:57 INFO Completed upload id=767008lb # [ 202.535319] niks3-server[1169]: 2026/09/22 11:34:57 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo7009worker1 # [ 202.577676] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=7527010worker1 # [ 202.587238] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345287011worker1 # [ 202.590215] nix-daemon[852]: reaped child process 2181, status = succeeded7012worker1 # [ 202.596197] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)7013lb # [ 202.559535] niks3-server[1169]: 2026/09/22 11:34:57 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls7014worker1 # [ 202.605946] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)7015worker1 # [ 204.035105] nix-daemon[852]: reaped child process 2195, status = succeeded7016worker1 # [ 204.145718] nix-daemon[852]: accepted connection from pid 2211, user nix-grpc-daemon (trusted)7017worker1 # [ 204.163124] nix-daemon[852]: reaped child process 2214, status = succeeded7018lb # [ 204.126644] niks3-server[1169]: 2026/09/22 11:34:58 INFO Received uploads request method=POST path=/api/pending_closures7019worker1 # [ 204.186492] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.780Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7020worker1 # [ 204.187962] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.781Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"7021lb # [ 204.172092] niks3-server[1169]: 2026/09/22 11:34:58 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv7022lb # [ 204.184708] niks3-server[1169]: 2026/09/22 11:34:58 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst7023worker1 # [ 204.238617] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.833Z level=INFO msg="Uploading 1 narinfos"7024lb # [ 204.196219] niks3-server[1169]: 2026/09/22 11:34:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7025lb # [ 204.199167] niks3-server[1169]: 2026/09/22 11:34:58 INFO Signed narinfos id=77 count=17026lb # [ 204.207865] niks3-server[1169]: 2026/09/22 11:34:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls7027lb # [ 204.215790] niks3-server[1169]: 2026/09/22 11:34:58 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7028lb # [ 204.221012] niks3-server[1169]: 2026/09/22 11:34:58 INFO Completed upload id=777029worker1 # [ 204.265180] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.860Z level=INFO msg="Upload complete. (244ms)"7030lb # [ 204.223470] niks3-server[1169]: 2026/09/22 11:34:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo7031worker2 # [ 204.278104] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:58Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=17032worker1 # [ 204.277144] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=17033worker1 # [ 204.295171] nix-daemon[852]: reaped child process 2193, status = succeeded7034worker1 # [ 204.317984] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345287035worker1 # [ 204.332539] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345287036worker1 # [ 204.347963] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.943Z level=INFO msg="All 2 paths already cached"7037worker1 # [ 204.358069] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345287038worker1 # [ 204.372782] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)7039worker1 # [ 204.523177] nix-daemon[852]: accepted connection from pid 2218, user nix-grpc-daemon (trusted)7040worker1 # [ 204.541757] nix-daemon[852]: reaped child process 2221, status = succeeded7041worker1 # [ 204.639522] nix-daemon[852]: accepted connection from pid 2223, user nix-grpc-daemon (trusted)7042worker1 # [ 204.657654] nix-daemon[852]: reaped child process 2226, status = succeeded7043lb # [ 204.610796] niks3-server[1169]: 2026/09/22 11:34:59 INFO Received uploads request method=POST path=/api/pending_closures7044worker1 # [ 204.666066] nix-grpc-daemon[1925]: time=2026-09-22T11:34:59.261Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7045worker1 # [ 204.667516] nix-grpc-daemon[1925]: time=2026-09-22T11:34:59.261Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"7046lb # [ 204.640046] niks3-server[1169]: 2026/09/22 11:34:59 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst7047worker1 # [ 204.697369] nix-grpc-daemon[1925]: time=2026-09-22T11:34:59.292Z level=INFO msg="Uploading 1 narinfos"7048lb # [ 204.653889] niks3-server[1169]: 2026/09/22 11:34:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7049lb # [ 204.663884] niks3-server[1169]: 2026/09/22 11:34:59 INFO Signed narinfos id=78 count=17050lb # [ 204.674493] niks3-server[1169]: 2026/09/22 11:34:59 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7051lb # [ 204.681518] niks3-server[1169]: 2026/09/22 11:34:59 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls7052worker1 # [ 204.732830] nix-grpc-daemon[1925]: time=2026-09-22T11:34:59.328Z level=INFO msg="Upload complete. (345ms)"7053lb # [ 204.687787] niks3-server[1169]: 2026/09/22 11:34:59 INFO Completed upload id=787054lb # [ 204.693555] niks3-server[1169]: 2026/09/22 11:34:59 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7055worker1 # [ 204.740128] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=7527056worker1 # [ 204.751843] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345287057worker1 # [ 204.757289] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)7058worker1 # [ 204.761802] nix-daemon[852]: reaped child process 2216, status = succeeded7059worker1 # [ 204.773674] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)7060worker1 # [ 205.832284] nix-daemon[852]: reaped child process 2230, status = succeeded7061worker1 # [ 205.916884] nix-daemon[852]: accepted connection from pid 2246, user nix-grpc-daemon (trusted)7062worker1 # [ 205.935029] nix-daemon[852]: reaped child process 2249, status = succeeded7063lb # [ 205.897927] niks3-server[1169]: 2026/09/22 11:35:00 INFO Received uploads request method=POST path=/api/pending_closures7064worker1 # [ 205.952964] nix-grpc-daemon[1925]: time=2026-09-22T11:35:00.548Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7065worker1 # [ 205.956507] nix-grpc-daemon[1925]: time=2026-09-22T11:35:00.550Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7066lb # [ 205.935612] niks3-server[1169]: 2026/09/22 11:35:00 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7067lb # [ 205.948338] niks3-server[1169]: 2026/09/22 11:35:00 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7068worker1 # [ 206.006462] nix-grpc-daemon[1925]: time=2026-09-22T11:35:00.601Z level=INFO msg="Uploading 1 narinfos"7069lb # [ 205.963095] niks3-server[1169]: 2026/09/22 11:35:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7070lb # [ 205.971809] niks3-server[1169]: 2026/09/22 11:35:00 INFO Signed narinfos id=79 count=17071lb # [ 205.978499] niks3-server[1169]: 2026/09/22 11:35:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7072lb # [ 205.987328] niks3-server[1169]: 2026/09/22 11:35:00 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7073lb # [ 206.002165] niks3-server[1169]: 2026/09/22 11:35:00 INFO Completed upload id=797074lb # [ 206.006961] niks3-server[1169]: 2026/09/22 11:35:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7075worker1 # [ 206.047345] nix-grpc-daemon[1925]: time=2026-09-22T11:35:00.642Z level=INFO msg="Upload complete. (264ms)"7076worker1 # [ 206.061353] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=17077worker1 # [ 206.079527] nix-daemon[852]: reaped child process 2228, status = succeeded7078client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.49 seconds)7079worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7080worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7081client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7082client # [ 206.276852] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7083client # Running as unit: gpu.service; invocation ID: 2b58976f03c54afba0b3d2e33a83e78e7084client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)7085lb: must succeed: curl -sf localhost:9901/clusters7086lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7087lb: must succeed: curl -sf localhost:9901/clusters7088lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7089worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7090worker2 # [ 206.463329] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351007091client # [ 206.488973] nix[1512]: don't know how to build these paths:7092client # [ 206.489977] nix[1512]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7093worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7094worker1 # [ 206.470530] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345287095client # [ 206.519008] nix[1512]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker7096worker2 # [ 206.499262] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:01Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07097lb: must succeed: curl -sf localhost:9901/clusters7098lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7099lb: must succeed: curl -sf localhost:9901/clusters7100lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7101worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7102worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7103client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7104lb # [ 208.335506] envoy[893]: conn peer=[2001:db8:1::1]:50956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27183 tx=12380 ms=182167105client # [ 236.525181] nix[1512]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}7106client # [ 236.536891] nix[1512]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7107client # [ 236.569243] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7108client # [ 236.577132] systemd[1]: gpu.service: Failed with result 'exit-code'.7109client # [ 236.580949] systemd[1]: gpu.service: Consumed 192ms CPU time over 30.298s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7110client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.08 seconds)7111client: must succeed: journalctl -u gpu -o cat7112client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7113lb: must succeed: curl -sf localhost:9901/clusters7114lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7115lb: must succeed: curl -sf localhost:9901/clusters7116lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7117worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7118worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7119(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.32 seconds)7120subtest: low disk drains a worker and builds go to the other7121worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7122worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.18 seconds)7123worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7124lb # [ 238.339333] envoy[893]: conn peer=[2001:db8:1::1]:42222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300667125worker1 # [ 238.386042] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:32Z level=info event=unhealthy reason=min_free7126worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)7127client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27128client # don't know how to build these paths:7129client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7130worker1 # [ 239.444589] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:345287131worker1 # [ 239.452175] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345287132worker2 # [ 239.481315] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07133worker2 # [ 239.488900] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17134worker2 # [ 239.533193] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351007135worker2 # [ 239.549404] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351007136client # copying 1 paths to 'lb:50051'...7137client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7138worker2 # [ 239.565824] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351007139worker2 # [ 239.583292] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7140worker2 # [ 239.747997] nix-daemon[842]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)7141worker2 # [ 239.765791] nix-daemon[842]: reaped child process 1865, status = succeeded7142worker2 # [ 239.863602] nix-daemon[842]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)7143worker2 # [ 239.877980] nix-daemon[842]: reaped child process 1870, status = succeeded7144lb # [ 239.837685] niks3-server[1169]: 2026/09/22 11:35:34 INFO Received uploads request method=POST path=/api/pending_closures7145worker2 # [ 239.898351] nix-grpc-daemon[1643]: time=2026-09-22T11:35:34.484Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7146worker2 # [ 239.899827] nix-grpc-daemon[1643]: time=2026-09-22T11:35:34.484Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7147lb # [ 239.880135] niks3-server[1169]: 2026/09/22 11:35:34 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7148worker2 # [ 239.942231] nix-grpc-daemon[1643]: time=2026-09-22T11:35:34.529Z level=INFO msg="Uploading 1 narinfos"7149lb # [ 239.893303] niks3-server[1169]: 2026/09/22 11:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7150lb # [ 239.903251] niks3-server[1169]: 2026/09/22 11:35:34 INFO Signed narinfos id=80 count=17151lb # [ 239.916398] niks3-server[1169]: 2026/09/22 11:35:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7152lb # [ 239.931789] niks3-server[1169]: 2026/09/22 11:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7153worker2 # [ 239.987925] nix-grpc-daemon[1643]: time=2026-09-22T11:35:34.574Z level=INFO msg="Upload complete. (387ms)"7154lb # [ 239.941274] niks3-server[1169]: 2026/09/22 11:35:34 INFO Completed upload id=807155lb # [ 239.946507] niks3-server[1169]: 2026/09/22 11:35:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7156worker2 # [ 240.001099] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=5127157worker2 # [ 240.016822] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351007158worker2 # [ 240.023889] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7159worker2 # [ 240.029887] nix-daemon[842]: reaped child process 1860, status = succeeded7160client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7161client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7162worker2 # [ 240.045455] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7163client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7164client # farm-a> LOG-farm-a-drain7165worker2 # [ 240.936808] nix-daemon[842]: reaped child process 1874, status = succeeded7166worker2 # [ 241.020457] nix-daemon[842]: accepted connection from pid 1890, user nix-grpc-daemon (trusted)7167worker2 # [ 241.037316] nix-daemon[842]: reaped child process 1893, status = succeeded7168lb # [ 240.997689] niks3-server[1169]: 2026/09/22 11:35:35 INFO Received uploads request method=POST path=/api/pending_closures7169worker2 # [ 241.059087] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.643Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7170worker2 # [ 241.060885] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.643Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7171lb # [ 241.036072] niks3-server[1169]: 2026/09/22 11:35:35 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7172lb # [ 241.046670] niks3-server[1169]: 2026/09/22 11:35:35 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7173worker2 # [ 241.111651] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.698Z level=INFO msg="Uploading 1 narinfos"7174lb # [ 241.057569] niks3-server[1169]: 2026/09/22 11:35:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7175lb # [ 241.067770] niks3-server[1169]: 2026/09/22 11:35:35 INFO Signed narinfos id=81 count=17176lb # [ 241.073403] niks3-server[1169]: 2026/09/22 11:35:35 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7177lb # [ 241.084140] niks3-server[1169]: 2026/09/22 11:35:35 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7178lb # [ 241.090573] niks3-server[1169]: 2026/09/22 11:35:35 INFO Completed upload id=817179worker2 # [ 241.139370] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.726Z level=INFO msg="Upload complete. (280ms)"7180lb # [ 241.094024] niks3-server[1169]: 2026/09/22 11:35:35 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7181worker2 # [ 241.158073] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=17182worker2 # [ 241.176316] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07183worker2 # [ 241.183078] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17184worker2 # [ 241.189160] nix-daemon[842]: reaped child process 1872, status = succeeded7185worker2 # [ 241.201117] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351007186worker2 # [ 241.216275] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351007187worker2 # [ 241.230746] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.817Z level=INFO msg="All 1 paths already cached"7188client # copying 1 paths to 'lb:50051'...7189client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7190worker2 # [ 241.241960] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351007191worker2 # [ 241.256134] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7192worker2 # [ 241.404053] nix-daemon[842]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)7193worker2 # [ 241.425811] nix-daemon[842]: reaped child process 1902, status = succeeded7194worker2 # [ 241.518177] nix-daemon[842]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)7195worker2 # [ 241.535647] nix-daemon[842]: reaped child process 1907, status = succeeded7196lb # [ 241.487774] niks3-server[1169]: 2026/09/22 11:35:36 INFO Received uploads request method=POST path=/api/pending_closures7197worker2 # [ 241.546554] nix-grpc-daemon[1643]: time=2026-09-22T11:35:36.133Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7198worker2 # [ 241.548075] nix-grpc-daemon[1643]: time=2026-09-22T11:35:36.133Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7199lb # [ 241.519127] niks3-server[1169]: 2026/09/22 11:35:36 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7200lb # [ 241.532269] niks3-server[1169]: 2026/09/22 11:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7201worker2 # [ 241.586190] nix-grpc-daemon[1643]: time=2026-09-22T11:35:36.173Z level=INFO msg="Uploading 1 narinfos"7202lb # [ 241.542878] niks3-server[1169]: 2026/09/22 11:35:36 INFO Signed narinfos id=82 count=17203lb # [ 241.547553] niks3-server[1169]: 2026/09/22 11:35:36 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7204lb # [ 241.555984] niks3-server[1169]: 2026/09/22 11:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7205lb # [ 241.560867] niks3-server[1169]: 2026/09/22 11:35:36 INFO Completed upload id=827206worker2 # [ 241.609049] nix-grpc-daemon[1643]: time=2026-09-22T11:35:36.196Z level=INFO msg="Upload complete. (338ms)"7207lb # [ 241.563122] niks3-server[1169]: 2026/09/22 11:35:36 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7208worker2 # [ 241.616183] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=6327209worker2 # [ 241.629684] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351007210worker2 # [ 241.639006] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7211worker2 # [ 241.641406] nix-daemon[842]: reaped child process 1897, status = succeeded7212client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7213client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7214worker2 # [ 241.654722] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7215client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7216client # farm-b> LOG-farm-b-drain7217worker2 # [ 242.708354] nix-daemon[842]: reaped child process 1911, status = succeeded7218worker2 # [ 242.803302] nix-daemon[842]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)7219worker2 # [ 242.821692] nix-daemon[842]: reaped child process 1930, status = succeeded7220lb # [ 242.778786] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received uploads request method=POST path=/api/pending_closures7221worker2 # [ 242.840479] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.424Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7222worker2 # [ 242.842282] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.424Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7223lb # [ 242.811116] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7224lb # [ 242.830065] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7225worker2 # [ 242.898016] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.484Z level=INFO msg="Uploading 1 narinfos"7226lb # [ 242.844627] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7227lb # [ 242.853760] niks3-server[1169]: 2026/09/22 11:35:37 INFO Signed narinfos id=83 count=17228lb # [ 242.855802] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7229lb # [ 242.867520] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7230lb # [ 242.872910] niks3-server[1169]: 2026/09/22 11:35:37 INFO Completed upload id=837231worker2 # [ 242.922116] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.509Z level=INFO msg="Upload complete. (258ms)"7232lb # [ 242.877428] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7233worker2 # [ 242.937491] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=17234worker2 # [ 242.959950] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07235worker2 # [ 242.965048] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17236worker2 # [ 242.969987] nix-daemon[842]: reaped child process 1909, status = succeeded7237worker2 # [ 242.997355] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351007238worker2 # [ 243.012322] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351007239worker2 # [ 243.026068] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.613Z level=INFO msg="All 2 paths already cached"7240client # copying 1 paths to 'lb:50051'...7241client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7242worker2 # [ 243.037312] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351007243worker2 # [ 243.051287] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7244worker2 # [ 243.209986] nix-daemon[842]: accepted connection from pid 1936, user nix-grpc-daemon (trusted)7245worker2 # [ 243.225106] nix-daemon[842]: reaped child process 1939, status = succeeded7246worker2 # [ 243.326543] nix-daemon[842]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)7247worker2 # [ 243.341541] nix-daemon[842]: reaped child process 1944, status = succeeded7248lb # [ 243.298622] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received uploads request method=POST path=/api/pending_closures7249worker2 # [ 243.356294] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.943Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7250worker2 # [ 243.357812] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.943Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7251lb # [ 243.328806] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7252worker2 # [ 243.389628] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.976Z level=INFO msg="Uploading 1 narinfos"7253lb # [ 243.342390] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7254lb # [ 243.353422] niks3-server[1169]: 2026/09/22 11:35:37 INFO Signed narinfos id=84 count=17255lb # [ 243.363410] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7256worker2 # [ 243.422419] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.009Z level=INFO msg="Upload complete. (355ms)"7257lb # [ 243.374960] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7258lb # [ 243.379570] niks3-server[1169]: 2026/09/22 11:35:37 INFO Completed upload id=847259worker2 # [ 243.432254] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=7607260lb # [ 243.384390] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7261worker2 # [ 243.443239] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351007262worker2 # [ 243.452648] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7263worker2 # [ 243.455614] nix-daemon[842]: reaped child process 1934, status = succeeded7264client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7265worker2 # [ 243.467085] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)7266client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7267client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7268client # farm-top> LOG-farm-top-drain7269worker2 # [ 244.100510] nix-daemon[842]: reaped child process 1948, status = succeeded7270worker2 # [ 244.199302] nix-daemon[842]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)7271worker2 # [ 244.216299] nix-daemon[842]: reaped child process 1967, status = succeeded7272lb # [ 244.174305] niks3-server[1169]: 2026/09/22 11:35:38 INFO Received uploads request method=POST path=/api/pending_closures7273worker2 # [ 244.232543] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.819Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7274worker2 # [ 244.235074] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.821Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7275lb # [ 244.212055] niks3-server[1169]: 2026/09/22 11:35:38 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7276lb # [ 244.222845] niks3-server[1169]: 2026/09/22 11:35:38 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7277worker2 # [ 244.285974] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.872Z level=INFO msg="Uploading 1 narinfos"7278lb # [ 244.237204] niks3-server[1169]: 2026/09/22 11:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7279lb # [ 244.245824] niks3-server[1169]: 2026/09/22 11:35:38 INFO Signed narinfos id=85 count=17280lb # [ 244.250665] niks3-server[1169]: 2026/09/22 11:35:38 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7281lb # [ 244.258483] niks3-server[1169]: 2026/09/22 11:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7282lb # [ 244.263711] niks3-server[1169]: 2026/09/22 11:35:38 INFO Completed upload id=857283worker2 # [ 244.312412] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.899Z level=INFO msg="Upload complete. (234ms)"7284lb # [ 244.266101] niks3-server[1169]: 2026/09/22 11:35:38 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7285worker2 # [ 244.325140] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=17286worker2 # [ 244.343508] nix-daemon[842]: reaped child process 1946, status = succeeded7287client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.14 seconds)7288client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7289client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7290worker1: must succeed: rm /nix/.rw-store/fill7291worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7292worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7293worker1 # [ 245.396449] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:39Z level=info event=healthy reason=min_free7294worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7295(finished: subtest: low disk drains a worker and builds go to the other, in 8.94 seconds)7296(finished: run the VM test script, in 246.76 seconds)7297test script finished in 247.00s7298cleanup7299kill QemuMachine (pid 49)7300client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7301client # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down7302client # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down7303client # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down7304kill QemuMachine (pid 50)7305lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7306lb # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down7307lb # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down7308lb # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down7309kill QemuMachine (pid 51)7310worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7311worker1 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down7312worker1 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down7313worker1 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down7314kill QemuMachine (pid 52)7315worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7316worker2 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down7317worker2 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down7318worker2 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down7319(finished: cleanup, in 1.24 seconds)7320additionally exposed symbols:7321 client, lb, worker1, worker2,7322 vlan1,7323 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh