vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #515
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13worker2 # Disk image does not exist, creating the virtualisation disk image...14lb: QEMU running (pid 49)15lb # Formatting '/build/vm-state-lb/tmp.7m614GNW7V', fmt=raw size=107374182416worker1 # Disk image does not exist, creating the virtualisation disk image...17worker2 # Formatting '/build/vm-state-worker2/tmp.zVyGKqOJw4', fmt=raw size=107374182418worker2: QEMU running (pid 52)19worker1: QEMU running (pid 51)20client # Disk image does not exist, creating the virtualisation disk image...21client: QEMU running (pid 50)22lb # mke2fs 1.47.4 (6-Mar-2025)23client # Formatting '/build/vm-state-client/tmp.hYmGd0DQ7u', fmt=raw size=107374182424worker1 # Formatting '/build/vm-state-worker1/tmp.oeOGQhh4r5', fmt=raw size=107374182425worker2 # mke2fs 1.47.4 (6-Mar-2025)26lb # Discarding device blocks: 0/262144 done27client # mke2fs 1.47.4 (6-Mar-2025)28worker2 # Discarding device blocks: 0/262144 done29lb # Creating filesystem with 262144 4k blocks and 65536 inodes30client # Discarding device blocks: 0/262144 done31worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes32lb # Filesystem UUID: 779349b4-bc2a-4469-83b4-5756cd0f9b6233client # Creating filesystem with 262144 4k blocks and 65536 inodes34worker2 # Filesystem UUID: 29033594-7ec8-4a6b-bbab-4875e6b0fb6035lb # Superblock backups stored on blocks:36client # Filesystem UUID: 3004aca3-c197-43f7-b97c-df9937b7960637worker2 # Superblock backups stored on blocks:38lb # 32768, 98304, 163840, 22937639client # Superblock backups stored on blocks:40worker2 # 32768, 98304, 163840, 22937641lb # 42client # 32768, 98304, 163840, 22937643worker2 # 44lb # Allocating group tables: 0/8 done45client # 46worker2 # Allocating group tables: 0/8 done47lb # Writing inode tables: 0/8 done48client # Allocating group tables: 0/8 done49worker2 # Writing inode tables: 0/8 done50lb # Creating journal (8192 blocks): done51client # Writing inode tables: 0/8 done52worker2 # Creating journal (8192 blocks): done53lb # Writing superblocks and filesystem accounting information: 0/8 done54client # Creating journal (8192 blocks): done55worker2 # Writing superblocks and filesystem accounting information: 0/8 done56lb # 57client # Writing superblocks and filesystem accounting information: 0/8 done58worker2 # 59lb # Virtualisation disk image created.60client # 61worker2 # Virtualisation disk image created.62lb # Starting virtiofs daemons...63client # Virtualisation disk image created.64worker2 # Starting virtiofs daemons...65lb # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)66client # Starting virtiofs daemons...67worker2 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)68lb # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether69client # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70worker2 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71lb # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...72client # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73worker2 # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...74lb # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75client # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...76worker2 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77lb # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78client # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79worker2 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether80lb # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...81client # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82worker2 # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...83lb # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84client # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...85worker2 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86lb # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87client # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88worker2 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89lb # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...90client # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether91worker1 # mke2fs 1.47.4 (6-Mar-2025)92(finished: start all VMs, in 0.50 seconds)93worker2 # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...94lb # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests95client # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker1 # Discarding device blocks: 0/262144 done97lb # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests98client # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests99worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes100lb # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests101client # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests102worker2 # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests103client: waiting for unit network-addresses-eth1.service104worker1 # Filesystem UUID: fd965096-981f-4865-952f-30cff7d8fd14105client: waiting for the VM to finish booting106worker1 # Superblock backups stored on blocks:107worker2 # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests108worker1 # 32768, 98304, 163840, 229376109worker2 # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests110worker1 # 111client # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests112worker1 # Allocating group tables: 0/8 done113worker1 # Writing inode tables: 0/8 done114worker1 # Creating journal (8192 blocks): done115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker1 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker1 # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker1 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker1 # [2026-09-23T04:43:09Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-23T04:43:09Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-23T04:43:09Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker1 # [ 0.000000] KASLR enabled134worker1 # [ 0.000000] random: crng init done135worker1 # [ 0.000000] Machine model: linux,dummy-virt136worker1 # [ 0.000000] efi: UEFI not found.137worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140worker1 # [ 0.000000] Zone ranges:141worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142worker1 # [ 0.000000] DMA32 empty143worker1 # [ 0.000000] Normal empty144worker1 # [ 0.000000] Device empty145worker1 # [ 0.000000] Movable zone start for each node146worker1 # [ 0.000000] Early memory node ranges147worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker1 # [ 0.000000] psci: probing for conduit method from DT.151worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153worker1 # [ 0.000000] psci: Trusted OS migration not required154worker1 # [ 0.000000] psci: SMC Calling Convention v1.1155worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157worker1 # [ 0.000000] Detected PIPT I-cache on CPU0158worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface160worker1 # [ 0.000000] CPU features: detected: Spectre-v4161worker1 # [ 0.000000] CPU features: detected: Spectre-BHB162worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164worker1 # [ 0.000000] alternatives: applying boot alternatives165worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/q94hnyyz97v2qgflmvps0svps2fkxgxc-nixos-system-worker1-test/init regInfo=/nix/store/g24d9rla8iffwzqri0ih21ibs5m7n8q1-closure-info/registration console=ttyAMA0,115200n8 console=tty0166lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]167worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/g24d9rla8iffwzqri0ih21ibs5m7n8q1-closure-info/registration", will be passed to user space.168lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026169lb # [ 0.000000] KASLR enabled170worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes171lb # [ 0.000000] random: crng init done172lb # [ 0.000000] Machine model: linux,dummy-virt173worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)174lb # [ 0.000000] efi: UEFI not found.175worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)176lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT177worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB178lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]179worker1 # [ 0.000000] software IO TLB: area num 1.180lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]181lb # [ 0.000000] Zone ranges:182worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB183lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]184worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)185lb # [ 0.000000] DMA32 empty186lb # [ 0.000000] Normal empty187worker1 # [ 0.000000] Fallback order for Node 0: 0188lb # [ 0.000000] Device empty189lb # [ 0.000000] Movable zone start for each node190worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216191worker1 # [ 0.000000] Policy zone: DMA192lb # [ 0.000000] Early memory node ranges193client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]194lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]195worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off196worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1197lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]198client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026199worker1 # [ 0.000000] allocated 3145728 bytes of page_ext200client # [ 0.000000] KASLR enabled201lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000202client # [ 0.000000] random: crng init done203worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages204lb # [ 0.000000] psci: probing for conduit method from DT.205client # [ 0.000000] Machine model: linux,dummy-virt206lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.207worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups208client # [ 0.000000] efi: UEFI not found.209worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.210lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs211client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT212worker1 # [ 0.000000] rcu: RCU event tracing is enabled.213lb # [ 0.000000] psci: Trusted OS migration not required214lb # [ 0.000000] psci: SMC Calling Convention v1.1215client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]216worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.217worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.218client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]219lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)220client # [ 0.000000] Zone ranges:221worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.222client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]223worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.224lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296225client # [ 0.000000] DMA32 empty226lb # [ 0.000000] Detected PIPT I-cache on CPU0227client # [ 0.000000] Normal empty228worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]229worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.230client # [ 0.000000] Device empty231lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)232client # [ 0.000000] Movable zone start for each node233worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1234client # [ 0.000000] Early memory node ranges235lb # [ 0.000000] CPU features: detected: GICv3 CPU interface236worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026237lb # [ 0.000000] CPU features: detected: Spectre-v4238worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.239client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]240worker2 # [ 0.000000] KASLR enabled241lb # [ 0.000000] CPU features: detected: Spectre-BHB242worker2 # [ 0.000000] random: crng init done243client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]244worker2 # [ 0.000000] Machine model: linux,dummy-virt245worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.246lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38247worker2 # [ 0.000000] efi: UEFI not found.248client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000249lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23250client # [ 0.000000] psci: probing for conduit method from DT.251worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT252worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.253lb # [ 0.000000] alternatives: applying boot alternatives254client # [ 0.000000] psci: PSCIv1.3 detected in firmware.255worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0256worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]257worker1 # [ 0.000000] GICv3: 256 SPIs implemented258client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs259worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented260worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]261client # [ 0.000000] psci: Trusted OS migration not required262worker2 # [ 0.000000] Zone ranges:263worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq264client # [ 0.000000] psci: SMC Calling Convention v1.1265worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]266worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI267worker2 # [ 0.000000] DMA32 empty268client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)269worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0270worker2 # [ 0.000000] Normal empty271worker2 # [ 0.000000] Device empty272lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test/init regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration console=ttyAMA0,115200n8 console=tty0273client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296274worker2 # [ 0.000000] Movable zone start for each node275worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000276worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]277worker2 # [ 0.000000] Early memory node ranges278client # [ 0.000000] Detected PIPT I-cache on CPU0279worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]280client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)281lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration", will be passed to user space.282worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)283worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]284client # [ 0.000000] CPU features: detected: GICv3 CPU interface285lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes286client # [ 0.000000] CPU features: detected: Spectre-v4287worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000288worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)289client # [ 0.000000] CPU features: detected: Spectre-BHB290worker2 # [ 0.000000] psci: probing for conduit method from DT.291lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)292worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000293worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.294client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38295lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)296worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000297worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs298client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23299worker2 # [ 0.000000] psci: Trusted OS migration not required300lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB301worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.302client # [ 0.000000] alternatives: applying boot alternatives303lb # [ 0.000000] software IO TLB: area num 1.304worker2 # [ 0.000000] psci: SMC Calling Convention v1.1305lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB306worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns307worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)308worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).309lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)310worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296311lb # [ 0.000000] Fallback order for Node 0: 0312worker2 # [ 0.000000] Detected PIPT I-cache on CPU0313worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns314lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216315lb # [ 0.000000] Policy zone: DMA316worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)317client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/y3p6raxm7zy0wsr4l77zwlv048rr50bg-nixos-system-client-test/init regInfo=/nix/store/w38x3dl7pam28qip5xmigpdq2yhn4g7y-closure-info/registration console=ttyAMA0,115200n8 console=tty0318worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns319worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface320lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off321worker1 # [ 0.000039] arm-pv: using stolen time PV322worker2 # [ 0.000000] CPU features: detected: Spectre-v4323lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1324worker2 # [ 0.000000] CPU features: detected: Spectre-BHB325client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/w38x3dl7pam28qip5xmigpdq2yhn4g7y-closure-info/registration", will be passed to user space.326lb # [ 0.000000] allocated 3145728 bytes of page_ext327worker1 # [ 0.000751] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)328worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38329lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages330worker1 # [ 0.000948] Console: colour dummy device 80x25331client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes332worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23333worker1 # [ 0.000956] printk: legacy console [tty0] enabled334lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups335worker2 # [ 0.000000] alternatives: applying boot alternatives336client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)337lb # [ 0.000000] rcu: Hierarchical RCU implementation.338lb # [ 0.000000] rcu: RCU event tracing is enabled.339worker1 # [ 0.001147] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)340client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)341worker1 # [ 0.001154] pid_max: default: 32768 minimum: 301342lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.343client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB344lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.345worker1 # [ 0.001247] LSM: initializing lsm=capability,landlock,yama,bpf,ima346client # [ 0.000000] software IO TLB: area num 1.347worker1 # [ 0.001458] landlock: Up and running.348lb # [ 0.000000] Rude variant of Tasks RCU enabled.349worker1 # [ 0.001461] Yama: becoming mindful.350client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB351lb # [ 0.000000] Tracing variant of Tasks RCU enabled.352worker1 # [ 0.002220] LSM support for eBPF active353worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n037bbz5rw2jqrqn5rpriyfa18g9m8j0-nixos-system-worker2-test/init regInfo=/nix/store/pmbjfiw1q9nphfwdak5m2czagw55f9yg-closure-info/registration console=ttyAMA0,115200n8 console=tty0354client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)355lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.356worker1 # [ 0.002383] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)357client # [ 0.000000] Fallback order for Node 0: 0358lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1359client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216360worker1 # [ 0.002418] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)361worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/pmbjfiw1q9nphfwdak5m2czagw55f9yg-closure-info/registration", will be passed to user space.362client # [ 0.000000] Policy zone: DMA363worker1 # [ 0.004384] rcu: Hierarchical SRCU implementation.364lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.365worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes366client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off367worker1 # [ 0.004390] rcu: Max phase no-delay instances is 1000.368lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.369worker1 # [ 0.005651] fsl-mc MSI: its@8080000 domain created370client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1371worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)372worker1 # [ 0.005742] EFI services will not be available.373client # [ 0.000000] allocated 3145728 bytes of page_ext374lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.375worker1 # [ 0.005817] smp: Bringing up secondary CPUs ...376worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)377client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages378worker1 # [ 0.005826] smp: Brought up 1 node, 1 CPU379lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0380worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB381client # [ 0.000000] ftrace: allocated 294 pages with 4 groups382lb # [ 0.000000] GICv3: 256 SPIs implemented383worker1 # [ 0.005829] SMP: Total of 1 processors activated.384worker2 # [ 0.000000] software IO TLB: area num 1.385client # [ 0.000000] rcu: Hierarchical RCU implementation.386worker1 # [ 0.005834] CPU: All CPU(s) started at EL1387lb # [ 0.000000] GICv3: 0 Extended SPIs implemented388client # [ 0.000000] rcu: RCU event tracing is enabled.389lb # [ 0.000000] Root IRQ handler: gic_handle_irq390worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB391worker1 # [ 0.005846] CPU features: detected: Branch Target Identification392lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI393client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.394worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)395worker1 # [ 0.005850] CPU features: detected: ARMv8.4 Translation Table Level396lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0397client # [ 0.000000] Trampoline variant of Tasks RCU enabled.398worker2 # [ 0.000000] Fallback order for Node 0: 0399client # [ 0.000000] Rude variant of Tasks RCU enabled.400lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000401worker1 # [ 0.005853] CPU features: detected: Instruction cache invalidation not required for I/D coherence402worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216403lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]404client # [ 0.000000] Tracing variant of Tasks RCU enabled.405worker2 # [ 0.000000] Policy zone: DMA406worker1 # [ 0.005857] CPU features: detected: Data cache clean to the PoU not required for I/D coherence407worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off408client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.409lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)410worker1 # [ 0.005861] CPU features: detected: Common not Private translations411worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1412client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1413worker1 # [ 0.005864] CPU features: detected: CRC32 instructions414worker2 # [ 0.000000] allocated 3145728 bytes of page_ext415lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)416worker1 # [ 0.005867] CPU features: detected: Data cache clean to Point of Deep Persistence417client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.418worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages419lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000420worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups421worker1 # [ 0.005871] CPU features: detected: Data cache clean to Point of Persistence422client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.423lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000424worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.425worker1 # [ 0.005874] CPU features: detected: Data independent timing control (DIT)426worker2 # [ 0.000000] rcu: RCU event tracing is enabled.427lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.428worker1 # [ 0.005877] CPU features: detected: E0PD429client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.430worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.431client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0432worker1 # [ 0.005879] CPU features: detected: Enhanced Counter Virtualization433lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns434worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.435client # [ 0.000000] GICv3: 256 SPIs implemented436worker1 # [ 0.005882] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)437worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.438lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).439client # [ 0.000000] GICv3: 0 Extended SPIs implemented440worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.441client # [ 0.000000] Root IRQ handler: gic_handle_irq442worker1 # [ 0.005886] CPU features: detected: Enhanced Virtualization Traps443client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI444worker1 # [ 0.005889] CPU features: detected: Fine Grained Traps445worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.446lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns447client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0448worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1449worker1 # [ 0.005893] CPU features: detected: Generic authentication (architected QARMA5 algorithm)450lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns451client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000452lb # [ 0.000032] arm-pv: using stolen time PV453worker1 # [ 0.005898] CPU features: detected: RCpc load-acquire (LDAPR)454client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]455worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.456worker1 # [ 0.005901] CPU features: detected: LSE atomic instructions457lb # [ 0.000500] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)458client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)459worker1 # [ 0.005904] CPU features: detected: Privileged Access Never460worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.461lb # [ 0.000713] Console: colour dummy device 80x25462worker1 # [ 0.005906] CPU features: detected: PMUv3463lb # [ 0.000721] printk: legacy console [tty0] enabled464worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.465worker1 # [ 0.005910] CPU features: detected: RAS Extension Support466client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)467worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0468worker1 # [ 0.005912] CPU features: detected: RASv1p1 Extension Support469client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000470lb # [ 0.000982] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)471worker2 # [ 0.000000] GICv3: 256 SPIs implemented472worker1 # [ 0.005915] CPU features: detected: Random Number Generator473lb # [ 0.000989] pid_max: default: 32768 minimum: 301474worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented475client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000476worker1 # [ 0.005918] CPU features: detected: Speculation barrier (SB)477worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq478lb # [ 0.001073] LSM: initializing lsm=capability,landlock,yama,bpf,ima479client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.480lb # [ 0.001254] landlock: Up and running.481worker1 # [ 0.005920] CPU features: detected: Stage-2 Force Write-Back482worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI483lb # [ 0.001257] Yama: becoming mindful.484worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0485worker1 # [ 0.005923] CPU features: detected: TLB range maintenance instructions486lb # [ 0.001787] LSM support for eBPF active487client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns488worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000489worker1 # [ 0.005928] CPU features: detected: Speculative Store Bypassing Safe (SSBS)490client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).491lb # [ 0.001932] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)492worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]493worker1 # [ 0.005968] alternatives: applying system-wide alternatives494lb # [ 0.001964] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)495client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns496worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)497worker1 # [ 0.009055] CPU features: detected: BBM Level 2 without TLB conflict abort498lb # [ 0.003741] rcu: Hierarchical SRCU implementation.499lb # [ 0.003747] rcu: Max phase no-delay instances is 1000.500client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns501worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)502lb # [ 0.005026] fsl-mc MSI: its@8080000 domain created503client # [ 0.000033] arm-pv: using stolen time PV504worker1 # [ 0.009272] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)505lb # [ 0.005126] EFI services will not be available.506worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000507worker1 # [ 0.009699] devtmpfs: initialized508lb # [ 0.005194] smp: Bringing up secondary CPUs ...509client # [ 0.000457] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)510worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000511lb # [ 0.005202] smp: Brought up 1 node, 1 CPU512worker1 # [ 0.011612] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)513client # [ 0.000650] Console: colour dummy device 80x25514lb # [ 0.005206] SMP: Total of 1 processors activated.515worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.516client # [ 0.000658] printk: legacy console [tty0] enabled517lb # [ 0.005211] CPU: All CPU(s) started at EL1518worker1 # [ 0.011634] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).519lb # [ 0.005221] CPU features: detected: Branch Target Identification520worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns521worker1 # [ 0.011818] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL522client # [ 0.000857] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)523worker1 # [ 0.011822] 0 pages in range for non-PLT usage524lb # [ 0.005225] CPU features: detected: ARMv8.4 Translation Table Level525worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).526client # [ 0.000864] pid_max: default: 32768 minimum: 301527worker1 # [ 0.011823] 508272 pages in range for PLT usage528client # [ 0.000938] LSM: initializing lsm=capability,landlock,yama,bpf,ima529worker1 # [ 0.011963] pinctrl core: initialized pinctrl subsystem530lb # [ 0.005228] CPU features: detected: Instruction cache invalidation not required for I/D coherence531worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns532client # [ 0.001094] landlock: Up and running.533worker1 # [ 0.012707] DMI not present or invalid.534client # [ 0.001097] Yama: becoming mindful.535lb # [ 0.005231] CPU features: detected: Data cache clean to the PoU not required for I/D coherence536worker1 # [ 0.016532] NET: Registered PF_NETLINK/PF_ROUTE protocol family537client # [ 0.001561] LSM support for eBPF active538worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns539worker2 # [ 0.000030] arm-pv: using stolen time PV540lb # [ 0.005235] CPU features: detected: Common not Private translations541worker1 # [ 0.019045] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations542client # [ 0.001720] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)543lb # [ 0.005239] CPU features: detected: CRC32 instructions544worker1 # [ 0.019249] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations545worker2 # [ 0.000480] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)546client # [ 0.001754] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)547lb # [ 0.005241] CPU features: detected: Data cache clean to Point of Deep Persistence548worker2 # [ 0.000657] Console: colour dummy device 80x25549client # [ 0.003714] rcu: Hierarchical SRCU implementation.550worker1 # [ 0.019453] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations551lb # [ 0.005245] CPU features: detected: Data cache clean to Point of Persistence552worker2 # [ 0.000664] printk: legacy console [tty0] enabled553client # [ 0.003720] rcu: Max phase no-delay instances is 1000.554worker1 # [ 0.019478] audit: initializing netlink subsys (disabled)555lb # [ 0.005248] CPU features: detected: Data independent timing control (DIT)556client # [ 0.005042] fsl-mc MSI: its@8080000 domain created557worker1 # [ 0.020108] thermal_sys: Registered thermal governor 'fair_share'558worker2 # [ 0.000853] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)559client # [ 0.005135] EFI services will not be available.560lb # [ 0.005251] CPU features: detected: E0PD561worker2 # [ 0.000860] pid_max: default: 32768 minimum: 301562client # [ 0.005227] smp: Bringing up secondary CPUs ...563worker1 # [ 0.020110] thermal_sys: Registered thermal governor 'bang_bang'564lb # [ 0.005254] CPU features: detected: Enhanced Counter Virtualization565client # [ 0.005236] smp: Brought up 1 node, 1 CPU566worker1 # [ 0.020114] thermal_sys: Registered thermal governor 'step_wise'567worker2 # [ 0.000931] LSM: initializing lsm=capability,landlock,yama,bpf,ima568lb # [ 0.005257] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)569client # [ 0.005239] SMP: Total of 1 processors activated.570worker2 # [ 0.001065] landlock: Up and running.571worker1 # [ 0.020117] thermal_sys: Registered thermal governor 'user_space'572worker2 # [ 0.001067] Yama: becoming mindful.573client # [ 0.005245] CPU: All CPU(s) started at EL1574lb # [ 0.005261] CPU features: detected: Enhanced Virtualization Traps575worker2 # [ 0.001563] LSM support for eBPF active576worker1 # [ 0.020122] thermal_sys: Registered thermal governor 'power_allocator'577client # [ 0.005256] CPU features: detected: Branch Target Identification578lb # [ 0.005264] CPU features: detected: Fine Grained Traps579worker2 # [ 0.001703] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)580client # [ 0.005261] CPU features: detected: ARMv8.4 Translation Table Level581worker1 # [ 0.020148] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1582lb # [ 0.005268] CPU features: detected: Generic authentication (architected QARMA5 algorithm)583worker1 # [ 0.020157] cpuidle: using governor ladder584worker2 # [ 0.001737] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)585lb # [ 0.005273] CPU features: detected: RCpc load-acquire (LDAPR)586worker1 # [ 0.020162] cpuidle: using governor menu587client # [ 0.005264] CPU features: detected: Instruction cache invalidation not required for I/D coherence588worker2 # [ 0.003656] rcu: Hierarchical SRCU implementation.589lb # [ 0.005276] CPU features: detected: LSE atomic instructions590worker1 # [ 0.020379] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.591worker2 # [ 0.003661] rcu: Max phase no-delay instances is 1000.592client # [ 0.005268] CPU features: detected: Data cache clean to the PoU not required for I/D coherence593lb # [ 0.005279] CPU features: detected: Privileged Access Never594worker1 # [ 0.020395] ASID allocator initialised with 65536 entries595worker2 # [ 0.004960] fsl-mc MSI: its@8080000 domain created596lb # [ 0.005281] CPU features: detected: PMUv3597client # [ 0.005272] CPU features: detected: Common not Private translations598worker1 # [ 0.021680] Serial: AMBA PL011 UART driver599worker2 # [ 0.005052] EFI services will not be available.600lb # [ 0.005284] CPU features: detected: RAS Extension Support601client # [ 0.005275] CPU features: detected: CRC32 instructions602worker2 # [ 0.005120] smp: Bringing up secondary CPUs ...603worker1 # [ 0.027110] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1604lb # [ 0.005287] CPU features: detected: RASv1p1 Extension Support605worker2 # [ 0.005129] smp: Brought up 1 node, 1 CPU606client # [ 0.005278] CPU features: detected: Data cache clean to Point of Deep Persistence607worker1 # [ 0.027276] printk: console [ttyAMA0] enabled608worker2 # [ 0.005133] SMP: Total of 1 processors activated.609lb # [ 0.005290] CPU features: detected: Random Number Generator610worker2 # [ 0.005138] CPU: All CPU(s) started at EL1611client # [ 0.005281] CPU features: detected: Data cache clean to Point of Persistence612lb # [ 0.005293] CPU features: detected: Speculation barrier (SB)613worker2 # [ 0.005149] CPU features: detected: Branch Target Identification614client # [ 0.005284] CPU features: detected: Data independent timing control (DIT)615lb # [ 0.005295] CPU features: detected: Stage-2 Force Write-Back616client # [ 0.005287] CPU features: detected: E0PD617worker2 # [ 0.005155] CPU features: detected: ARMv8.4 Translation Table Level618lb # [ 0.005298] CPU features: detected: TLB range maintenance instructions619client # [ 0.005290] CPU features: detected: Enhanced Counter Virtualization620lb # [ 0.005303] CPU features: detected: Speculative Store Bypassing Safe (SSBS)621worker2 # [ 0.005158] CPU features: detected: Instruction cache invalidation not required for I/D coherence622client # [ 0.005293] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)623lb # [ 0.005342] alternatives: applying system-wide alternatives624worker2 # [ 0.005162] CPU features: detected: Data cache clean to the PoU not required for I/D coherence625client # [ 0.005297] CPU features: detected: Enhanced Virtualization Traps626lb # [ 0.008323] CPU features: detected: BBM Level 2 without TLB conflict abort627worker2 # [ 0.005165] CPU features: detected: Common not Private translations628client # [ 0.005300] CPU features: detected: Fine Grained Traps629worker2 # [ 0.005169] CPU features: detected: CRC32 instructions630client # [ 0.005303] CPU features: detected: Generic authentication (architected QARMA5 algorithm)631lb # [ 0.008552] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)632worker2 # [ 0.005172] CPU features: detected: Data cache clean to Point of Deep Persistence633client # [ 0.005307] CPU features: detected: RCpc load-acquire (LDAPR)634lb # [ 0.008955] devtmpfs: initialized635worker2 # [ 0.005175] CPU features: detected: Data cache clean to Point of Persistence636client # [ 0.005310] CPU features: detected: LSE atomic instructions637lb # [ 0.010819] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)638worker1 # [ 0.146562] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages639worker2 # [ 0.005178] CPU features: detected: Data independent timing control (DIT)640client # [ 0.005313] CPU features: detected: Privileged Access Never641client # [ 0.005316] CPU features: detected: PMUv3642worker2 # [ 0.005181] CPU features: detected: E0PD643lb # [ 0.010841] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).644worker1 # [ 0.146579] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page645client # [ 0.005319] CPU features: detected: RAS Extension Support646worker2 # [ 0.005184] CPU features: detected: Enhanced Counter Virtualization647lb # [ 0.011030] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL648worker1 # [ 0.146585] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages649client # [ 0.005322] CPU features: detected: RASv1p1 Extension Support650lb # [ 0.011034] 0 pages in range for non-PLT usage651worker2 # [ 0.005187] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)652worker1 # [ 0.146589] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page653client # [ 0.005324] CPU features: detected: Random Number Generator654lb # [ 0.011035] 508272 pages in range for PLT usage655worker2 # [ 0.005191] CPU features: detected: Enhanced Virtualization Traps656worker1 # [ 0.146594] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages657client # [ 0.005327] CPU features: detected: Speculation barrier (SB)658lb # [ 0.011151] pinctrl core: initialized pinctrl subsystem659worker2 # [ 0.005194] CPU features: detected: Fine Grained Traps660lb # [ 0.011933] DMI not present or invalid.661client # [ 0.005329] CPU features: detected: Stage-2 Force Write-Back662worker1 # [ 0.146598] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page663lb # [ 0.015102] NET: Registered PF_NETLINK/PF_ROUTE protocol family664worker2 # [ 0.005198] CPU features: detected: Generic authentication (architected QARMA5 algorithm)665client # [ 0.005332] CPU features: detected: TLB range maintenance instructions666worker1 # [ 0.146603] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages667worker2 # [ 0.005204] CPU features: detected: RCpc load-acquire (LDAPR)668lb # [ 0.017621] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations669worker1 # [ 0.146607] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page670client # [ 0.005337] CPU features: detected: Speculative Store Bypassing Safe (SSBS)671worker2 # [ 0.005207] CPU features: detected: LSE atomic instructions672client # [ 0.005377] alternatives: applying system-wide alternatives673lb # [ 0.017822] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations674worker2 # [ 0.005210] CPU features: detected: Privileged Access Never675worker2 # [ 0.005213] CPU features: detected: PMUv3676client # [ 0.008439] CPU features: detected: BBM Level 2 without TLB conflict abort677lb # [ 0.018026] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations678worker2 # [ 0.005216] CPU features: detected: RAS Extension Support679lb # [ 0.018048] audit: initializing netlink subsys (disabled)680worker1 # [ 0.153897] fbcon: Taking over console681worker1 # [ 0.153913] ACPI: Interpreter disabled.682worker2 # [ 0.005219] CPU features: detected: RASv1p1 Extension Support683lb # [ 0.018603] thermal_sys: Registered thermal governor 'fair_share'684client # [ 0.008602] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)685worker2 # [ 0.005221] CPU features: detected: Random Number Generator686client # [ 0.009033] devtmpfs: initialized687lb # [ 0.018604] thermal_sys: Registered thermal governor 'bang_bang'688worker2 # [ 0.005224] CPU features: detected: Speculation barrier (SB)689lb # [ 0.018608] thermal_sys: Registered thermal governor 'step_wise'690client # [ 0.010896] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)691worker2 # [ 0.005227] CPU features: detected: Stage-2 Force Write-Back692lb # [ 0.018611] thermal_sys: Registered thermal governor 'user_space'693client # [ 0.010919] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).694worker2 # [ 0.005230] CPU features: detected: TLB range maintenance instructions695client # [ 0.011130] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL696worker2 # [ 0.005235] CPU features: detected: Speculative Store Bypassing Safe (SSBS)697lb # [ 0.018615] thermal_sys: Registered thermal governor 'power_allocator'698client # [ 0.011134] 0 pages in range for non-PLT usage699worker2 # [ 0.005274] alternatives: applying system-wide alternatives700client # [ 0.011135] 508272 pages in range for PLT usage701lb # [ 0.018641] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1702worker2 # [ 0.008291] CPU features: detected: BBM Level 2 without TLB conflict abort703client # [ 0.011263] pinctrl core: initialized pinctrl subsystem704lb # [ 0.018649] cpuidle: using governor ladder705client # [ 0.012081] DMI not present or invalid.706lb # [ 0.018655] cpuidle: using governor menu707client # [ 0.015680] NET: Registered PF_NETLINK/PF_ROUTE protocol family708lb # [ 0.018844] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.709worker2 # [ 0.008492] Memory: 1405680K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)710worker2 # [ 0.008932] devtmpfs: initialized711client # [ 0.018094] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations712lb # [ 0.018860] ASID allocator initialised with 65536 entries713lb # [ 0.020061] Serial: AMBA PL011 UART driver714worker2 # [ 0.010888] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)715client # [ 0.018277] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations716worker1 # [ 0.162968] iommu: Default domain type: Translated717lb # [ 0.025477] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1718worker2 # [ 0.010912] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).719client # [ 0.018483] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations720worker1 # [ 0.162979] iommu: DMA domain TLB invalidation policy: strict mode721lb # [ 0.025644] printk: console [ttyAMA0] enabled722worker1 # [ 0.163331] SCSI subsystem initialized723client # [ 0.018506] audit: initializing netlink subsys (disabled)724client # [ 0.019167] thermal_sys: Registered thermal governor 'fair_share'725worker2 # [ 0.011115] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL726worker1 # [ 0.165296] usbcore: registered new interface driver usbfs727worker2 # [ 0.011119] 0 pages in range for non-PLT usage728client # [ 0.019169] thermal_sys: Registered thermal governor 'bang_bang'729worker2 # [ 0.011120] 508272 pages in range for PLT usage730worker1 # [ 0.165329] usbcore: registered new interface driver hub731client # [ 0.019173] thermal_sys: Registered thermal governor 'step_wise'732worker2 # [ 0.011244] pinctrl core: initialized pinctrl subsystem733worker1 # [ 0.165347] usbcore: registered new device driver usb734client # [ 0.019176] thermal_sys: Registered thermal governor 'user_space'735worker2 # [ 0.012029] DMI not present or invalid.736worker1 # [ 0.165617] pps_core: LinuxPPS API ver. 1 registered737worker2 # [ 0.015562] NET: Registered PF_NETLINK/PF_ROUTE protocol family738client # [ 0.019182] thermal_sys: Registered thermal governor 'power_allocator'739worker1 # [ 0.165623] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>740worker2 # [ 0.017998] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations741client # [ 0.019213] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1742worker1 # [ 0.165634] PTP clock support registered743worker1 # [ 0.165685] EDAC MC: Ver: 3.0.0744client # [ 0.019221] cpuidle: using governor ladder745worker2 # [ 0.018183] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations746client # [ 0.019227] cpuidle: using governor menu747worker1 # [ 0.170292] scmi_core: SCMI protocol bus registered748worker2 # [ 0.018387] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations749client # [ 0.019425] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.750lb # [ 0.143208] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages751worker2 # [ 0.018411] audit: initializing netlink subsys (disabled)752client # [ 0.019442] ASID allocator initialised with 65536 entries753lb # [ 0.143225] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page754worker1 # [ 0.171296] FPGA manager framework755worker1 # [ 0.172261] vgaarb: loaded756client # [ 0.020709] Serial: AMBA PL011 UART driver757worker2 # [ 0.019030] thermal_sys: Registered thermal governor 'fair_share'758lb # [ 0.143231] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages759worker2 # [ 0.019032] thermal_sys: Registered thermal governor 'bang_bang'760worker1 # [ 0.172876] clocksource: Switched to clocksource arch_sys_counter761client # [ 0.026134] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1762lb # [ 0.143235] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page763client # [ 0.026304] printk: console [ttyAMA0] enabled764worker2 # [ 0.019038] thermal_sys: Registered thermal governor 'step_wise'765lb # [ 0.143239] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages766worker2 # [ 0.019042] thermal_sys: Registered thermal governor 'user_space'767lb # [ 0.143244] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page768worker2 # [ 0.019048] thermal_sys: Registered thermal governor 'power_allocator'769lb # [ 0.143248] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages770worker2 # [ 0.019101] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1771lb # [ 0.143252] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page772worker2 # [ 0.019110] cpuidle: using governor ladder773worker1 # [ 0.176725] VFS: Disk quotas dquot_6.6.0774worker2 # [ 0.019116] cpuidle: using governor menu775worker1 # [ 0.176770] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)776worker2 # [ 0.019309] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.777lb # [ 0.150726] fbcon: Taking over console778worker2 # [ 0.019327] ASID allocator initialised with 65536 entries779lb # [ 0.150742] ACPI: Interpreter disabled.780worker2 # [ 0.020581] Serial: AMBA PL011 UART driver781worker2 # [ 0.025926] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1782worker2 # [ 0.026088] printk: console [ttyAMA0] enabled783worker1 # [ 0.180555] netfs: FS-Cache loaded784worker1 # [ 0.180682] pnp: PnP ACPI: disabled785client # [ 0.144871] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages786client # [ 0.144901] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page787client # [ 0.144909] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages788client # [ 0.144913] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page789client # [ 0.144918] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages790worker1 # [ 0.184699] NET: Registered PF_INET protocol family791client # [ 0.144922] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page792client # [ 0.144926] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages793client # [ 0.144931] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page794lb # [ 0.159770] iommu: Default domain type: Translated795lb # [ 0.159780] iommu: DMA domain TLB invalidation policy: strict mode796lb # [ 0.160175] SCSI subsystem initialized797client # [ 0.152661] fbcon: Taking over console798worker1 # [ 0.185613] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)799client # [ 0.152676] ACPI: Interpreter disabled.800lb # [ 0.162214] usbcore: registered new interface driver usbfs801worker2 # [ 0.146646] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages802lb # [ 0.162248] usbcore: registered new interface driver hub803worker2 # [ 0.146665] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page804lb # [ 0.162263] usbcore: registered new device driver usb805lb # [ 0.162539] pps_core: LinuxPPS API ver. 1 registered806worker2 # [ 0.146670] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages807worker2 # [ 0.146674] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page808lb # [ 0.162546] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>809lb # [ 0.162556] PTP clock support registered810worker2 # [ 0.146679] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages811lb # [ 0.162606] EDAC MC: Ver: 3.0.0812worker2 # [ 0.146683] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page813lb # [ 0.167431] scmi_core: SCMI protocol bus registered814worker2 # [ 0.146687] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages815lb # [ 0.168438] FPGA manager framework816worker2 # [ 0.146691] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page817lb # [ 0.169412] vgaarb: loaded818lb # [ 0.170077] clocksource: Switched to clocksource arch_sys_counter819client # [ 0.161688] iommu: Default domain type: Translated820worker2 # [ 0.154406] fbcon: Taking over console821client # [ 0.161700] iommu: DMA domain TLB invalidation policy: strict mode822worker2 # [ 0.154422] ACPI: Interpreter disabled.823client # [ 0.162069] SCSI subsystem initialized824worker2 # [ 0.156362] iommu: Default domain type: Translated825client # [ 0.164092] usbcore: registered new interface driver usbfs826worker2 # [ 0.156372] iommu: DMA domain TLB invalidation policy: strict mode827client # [ 0.164125] usbcore: registered new interface driver hub828worker2 # [ 0.158114] SCSI subsystem initialized829client # [ 0.164140] usbcore: registered new device driver usb830client # [ 0.164437] pps_core: LinuxPPS API ver. 1 registered831client # [ 0.164443] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>832client # [ 0.164453] PTP clock support registered833lb # [ 0.176648] VFS: Disk quotas dquot_6.6.0834client # [ 0.164502] EDAC MC: Ver: 3.0.0835lb # [ 0.176693] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)836client # [ 0.169214] scmi_core: SCMI protocol bus registered837lb # [ 0.178411] netfs: FS-Cache loaded838lb # [ 0.178569] pnp: PnP ACPI: disabled839client # [ 0.170249] FPGA manager framework840client # [ 0.171232] vgaarb: loaded841worker2 # [ 0.163181] usbcore: registered new interface driver usbfs842client # [ 0.171889] clocksource: Switched to clocksource arch_sys_counter843worker2 # [ 0.163214] usbcore: registered new interface driver hub844worker2 # [ 0.163230] usbcore: registered new device driver usb845worker2 # [ 0.163508] pps_core: LinuxPPS API ver. 1 registered846worker2 # [ 0.163514] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>847worker2 # [ 0.163525] PTP clock support registered848worker2 # [ 0.163580] EDAC MC: Ver: 3.0.0849worker2 # [ 0.168307] scmi_core: SCMI protocol bus registered850client # [ 0.176293] VFS: Disk quotas dquot_6.6.0851client # [ 0.176334] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)852worker2 # [ 0.169312] FPGA manager framework853lb # [ 0.182575] NET: Registered PF_INET protocol family854worker2 # [ 0.170321] vgaarb: loaded855lb # [ 0.182849] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)856worker2 # [ 0.170989] clocksource: Switched to clocksource arch_sys_counter857client # [ 0.180187] netfs: FS-Cache loaded858client # [ 0.180341] pnp: PnP ACPI: disabled859worker2 # [ 0.174980] VFS: Disk quotas dquot_6.6.0860worker1 # [ 0.214585] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)861worker2 # [ 0.177625] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)862worker2 # [ 0.178876] netfs: FS-Cache loaded863worker1 # [ 0.214639] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)864worker2 # [ 0.178983] pnp: PnP ACPI: disabled865worker1 # [ 0.214671] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)866client # [ 0.184371] NET: Registered PF_INET protocol family867worker1 # [ 0.214748] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)868client # [ 0.184647] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)869worker1 # [ 0.214890] TCP: Hash tables configured (established 16384 bind 16384)870worker1 # [ 0.214972] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)871worker2 # [ 0.182959] NET: Registered PF_INET protocol family872worker1 # [ 0.215008] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)873worker1 # [ 0.215044] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)874worker1 # [ 0.215144] NET: Registered PF_UNIX/PF_LOCAL protocol family875worker1 # [ 0.215166] NET: Registered PF_XDP protocol family876worker1 # [ 0.215184] PCI: CLS 0 bytes, default 64877worker1 # [ 0.215444] Trying to unpack rootfs image as initramfs...878worker2 # [ 0.183892] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)879worker1 # [ 0.230772] kvm [1]: HYP mode not available880lb # [ 0.213192] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881lb # [ 0.213241] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882lb # [ 0.213271] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883client # [ 0.215547] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)884lb # [ 0.213355] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)885client # [ 0.215601] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)886lb # [ 0.213497] TCP: Hash tables configured (established 16384 bind 16384)887client # [ 0.215631] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)888lb # [ 0.213582] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)889client # [ 0.215715] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)890lb # [ 0.213618] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)891client # [ 0.215859] TCP: Hash tables configured (established 16384 bind 16384)892lb # [ 0.213655] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)893lb # [ 0.213800] NET: Registered PF_UNIX/PF_LOCAL protocol family894lb # [ 0.213839] NET: Registered PF_XDP protocol family895lb # [ 0.213858] PCI: CLS 0 bytes, default 64896lb # [ 0.214132] Trying to unpack rootfs image as initramfs...897client # [ 0.220923] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)898client # [ 0.220965] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)899lb # [ 0.232081] kvm [1]: HYP mode not available900client # [ 0.221002] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)901client # [ 0.221130] NET: Registered PF_UNIX/PF_LOCAL protocol family902client # [ 0.221185] NET: Registered PF_XDP protocol family903worker2 # [ 0.213727] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)904client # [ 0.221205] PCI: CLS 0 bytes, default 64905worker2 # [ 0.213780] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)906client # [ 0.221464] Trying to unpack rootfs image as initramfs...907worker2 # [ 0.213810] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)908worker2 # [ 0.213882] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)909worker2 # [ 0.214028] TCP: Hash tables configured (established 16384 bind 16384)910worker2 # [ 0.214118] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)911worker2 # [ 0.214156] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)912worker2 # [ 0.214193] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)913worker2 # [ 0.214308] NET: Registered PF_UNIX/PF_LOCAL protocol family914worker2 # [ 0.214336] NET: Registered PF_XDP protocol family915worker2 # [ 0.214354] PCI: CLS 0 bytes, default 64916client # [ 0.233058] kvm [1]: HYP mode not available917worker2 # [ 0.214602] Trying to unpack rootfs image as initramfs...918worker2 # [ 0.232913] kvm [1]: HYP mode not available919worker1 # [ 0.328983] Initialise system trusted keyrings920worker1 # [ 0.329759] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker1 # [ 0.337405] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker1 # [ 0.338241] 9p: Installing v9fs 9p2000 file system support923worker1 # [ 0.358217] Key type asymmetric registered924worker1 # [ 0.358243] Asymmetric key parser 'x509' registered925worker1 # [ 0.358338] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)926worker1 # [ 0.360501] io scheduler mq-deadline registered927worker1 # [ 0.360513] io scheduler kyber registered928lb # [ 0.338698] Initialise system trusted keyrings929lb # [ 0.339467] workingset: timestamp_bits=42 max_order=19 bucket_order=0930lb # [ 0.340755] squashfs: version 4.0 (2009/01/31) Phillip Lougher931lb # [ 0.341535] 9p: Installing v9fs 9p2000 file system support932worker1 # [ 0.374059] pl061_gpio 9030000.pl061: PL061 GPIO chip registered933worker2 # [ 0.332357] Initialise system trusted keyrings934client # [ 0.340477] Initialise system trusted keyrings935worker2 # [ 0.333258] workingset: timestamp_bits=42 max_order=19 bucket_order=0936worker1 # [ 0.375408] ledtrig-cpu: registered to indicate activity on CPUs937client # [ 0.341254] workingset: timestamp_bits=42 max_order=19 bucket_order=0938worker1 # [ 0.375785] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:939worker2 # [ 0.334539] squashfs: version 4.0 (2009/01/31) Phillip Lougher940client # [ 0.342546] squashfs: version 4.0 (2009/01/31) Phillip Lougher941worker1 # [ 0.375802] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000942client # [ 0.343329] 9p: Installing v9fs 9p2000 file system support943worker1 # [ 0.375823] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000944worker1 # [ 0.375832] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000945worker1 # [ 0.375860] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits946worker2 # [ 0.335405] 9p: Installing v9fs 9p2000 file system support947worker1 # [ 0.375889] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]948worker1 # [ 0.375985] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00949worker1 # [ 0.375995] pci_bus 0000:00: root bus resource [bus 00-ff]950worker1 # [ 0.376002] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]951worker1 # [ 0.376007] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]952worker1 # [ 0.376012] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]953worker1 # [ 0.376067] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint954worker1 # [ 0.376505] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint955worker1 # [ 0.376694] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]956worker1 # [ 0.376711] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]957worker1 # [ 0.376740] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]958worker1 # [ 0.376757] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]959worker1 # [ 0.393158] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint960lb # [ 0.370269] Key type asymmetric registered961worker1 # [ 0.393346] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]962lb # [ 0.370300] Asymmetric key parser 'x509' registered963worker1 # [ 0.393363] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]964lb # [ 0.370381] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)965worker1 # [ 0.393393] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]966lb # [ 0.372604] io scheduler mq-deadline registered967worker1 # [ 0.393842] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint968lb # [ 0.372616] io scheduler kyber registered969worker1 # [ 0.394031] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]970worker1 # [ 0.394047] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]971worker1 # [ 0.394077] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]972worker1 # [ 0.394520] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint973worker1 # [ 0.394701] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]974worker1 # [ 0.394717] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]975worker1 # [ 0.394747] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]976worker1 # [ 0.394767] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]977worker2 # [ 0.363450] Key type asymmetric registered978worker1 # [ 0.395212] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint979worker2 # [ 0.363482] Asymmetric key parser 'x509' registered980client # [ 0.372127] Key type asymmetric registered981worker1 # [ 0.395395] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]982client # [ 0.372153] Asymmetric key parser 'x509' registered983worker2 # [ 0.363560] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)984worker1 # [ 0.395426] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]985worker2 # [ 0.365762] io scheduler mq-deadline registered986client # [ 0.372246] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)987worker2 # [ 0.365773] io scheduler kyber registered988client # [ 0.374355] io scheduler mq-deadline registered989worker1 # [ 0.395881] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint990lb # [ 0.382227] pl061_gpio 9030000.pl061: PL061 GPIO chip registered991client # [ 0.374366] io scheduler kyber registered992worker1 # [ 0.396073] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]993worker1 # [ 0.396103] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker1 # [ 0.396527] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint995lb # [ 0.383665] ledtrig-cpu: registered to indicate activity on CPUs996worker1 # [ 0.396710] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]997lb # [ 0.384076] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:998lb # [ 0.384094] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000999lb # [ 0.384119] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001000lb # [ 0.384128] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001001lb # [ 0.384149] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1002worker2 # [ 0.375159] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1003lb # [ 0.384175] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1004client # [ 0.384048] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1005lb # [ 0.384256] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001006lb # [ 0.384266] pci_bus 0000:00: root bus resource [bus 00-ff]1007lb # [ 0.384272] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1008worker2 # [ 0.376675] ledtrig-cpu: registered to indicate activity on CPUs1009lb # [ 0.384276] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1010worker2 # [ 0.377097] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1011client # [ 0.385556] ledtrig-cpu: registered to indicate activity on CPUs1012lb # [ 0.384281] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1013worker2 # [ 0.377116] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001014client # [ 0.385971] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1015lb # [ 0.384345] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1016worker1 # [ 0.421038] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1017worker2 # [ 0.377128] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001018client # [ 0.385988] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001019worker1 # [ 0.421252] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1020lb # [ 0.384821] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1021lb # [ 0.385015] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1022client # [ 0.386000] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001023worker2 # [ 0.377136] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001024worker1 # [ 0.421284] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025lb # [ 0.385032] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1026worker2 # [ 0.377156] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1027worker1 # [ 0.421793] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1028client # [ 0.386009] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001029lb # [ 0.385063] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1030worker1 # [ 0.421982] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1031worker2 # [ 0.377184] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1032lb # [ 0.385081] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1033client # [ 0.386030] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1034worker1 # [ 0.422013] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035worker2 # [ 0.377260] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001036lb # [ 0.385556] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1037client # [ 0.386055] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1038worker2 # [ 0.377278] pci_bus 0000:00: root bus resource [bus 00-ff]1039worker1 # [ 0.422482] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1040lb # [ 0.385750] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1041client # [ 0.386133] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001042worker1 # [ 0.422670] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1043worker2 # [ 0.377285] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1044lb # [ 0.385767] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1045client # [ 0.386142] pci_bus 0000:00: root bus resource [bus 00-ff]1046worker1 # [ 0.422700] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1047worker2 # [ 0.377291] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1048lb # [ 0.385797] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049client # [ 0.386148] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1050worker2 # [ 0.377296] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1051worker1 # [ 0.423172] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1052client # [ 0.386153] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1053worker1 # [ 0.423458] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1054worker2 # [ 0.377350] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1055client # [ 0.386158] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1056worker1 # [ 0.423476] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1057worker2 # [ 0.377790] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1058worker1 # [ 0.423506] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059client # [ 0.386245] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1060worker2 # [ 0.377976] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1061worker1 # [ 0.423987] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1062client # [ 0.386690] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1063worker2 # [ 0.377993] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1064worker1 # [ 0.424172] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1065client # [ 0.386877] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1066worker2 # [ 0.378022] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067worker1 # [ 0.424188] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1068client # [ 0.386893] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker2 # [ 0.378038] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1070lb # [ 0.405435] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1071worker1 # [ 0.424219] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1072client # [ 0.386923] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1073worker2 # [ 0.378499] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1074lb # [ 0.405629] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1075client # [ 0.386939] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1076worker1 # [ 0.424808] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1077worker2 # [ 0.378683] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1078lb # [ 0.405647] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1079worker1 # [ 0.424820] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1080client # [ 0.387398] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1081worker2 # [ 0.378699] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1082lb # [ 0.405677] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1083client # [ 0.387580] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1084worker1 # [ 0.424825] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1085worker2 # [ 0.378728] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086lb # [ 0.406157] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1087client # [ 0.387596] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1088worker1 # [ 0.424871] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1089lb # [ 0.406342] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1090client # [ 0.387625] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1091lb # [ 0.406358] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1092lb # [ 0.406388] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093lb # [ 0.406406] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1094client # [ 0.407030] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1095client # [ 0.407218] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1096lb # [ 0.406872] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1097client # [ 0.407235] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1098lb # [ 0.407062] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1099client # [ 0.407264] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100lb # [ 0.407093] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101client # [ 0.407722] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1102lb # [ 0.407568] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1103lb # [ 0.407759] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1104lb # [ 0.407789] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1105lb # [ 0.408182] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1106lb # [ 0.408364] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1107lb # [ 0.408637] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1108lb # [ 0.408827] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1109worker1 # [ 0.448998] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1110lb # [ 0.408857] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111worker1 # [ 0.449061] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1112lb # [ 0.409316] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1113worker2 # [ 0.403295] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1114lb # [ 0.409503] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1115worker1 # [ 0.449111] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1116worker2 # [ 0.403509] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1117lb # [ 0.409534] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118worker2 # [ 0.403528] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1119worker1 # [ 0.449159] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1120lb # [ 0.409999] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1121worker2 # [ 0.403558] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122worker1 # [ 0.449207] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1123worker2 # [ 0.404030] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1124worker1 # [ 0.449255] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1125worker2 # [ 0.404217] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1126worker1 # [ 0.449302] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1127worker2 # [ 0.404232] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1128worker1 # [ 0.449349] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1129worker2 # [ 0.404263] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1130worker2 # [ 0.404282] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1131worker1 # [ 0.449473] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1132worker2 # [ 0.404756] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1133worker1 # [ 0.449523] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1134worker2 # [ 0.404945] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1135worker1 # [ 0.449547] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1136worker2 # [ 0.404975] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137client # [ 0.419999] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1138worker1 # [ 0.449568] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1139client # [ 0.420034] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1140worker2 # [ 0.405434] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1141worker1 # [ 0.449590] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1142client # [ 0.420064] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143worker2 # [ 0.405621] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1144worker1 # [ 0.449612] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1145client # [ 0.420083] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1146worker2 # [ 0.405650] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147worker1 # [ 0.449633] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1148client # [ 0.420632] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1149worker1 # [ 0.449656] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1150worker2 # [ 0.406074] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1151client # [ 0.420829] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1152worker1 # [ 0.449684] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1153worker2 # [ 0.406256] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1154client # [ 0.420859] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1155worker1 # [ 0.449706] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1156worker2 # [ 0.406514] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1157worker1 # [ 0.449728] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1158client # [ 0.421321] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1159worker2 # [ 0.406704] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1160lb # [ 0.438274] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1161client # [ 0.421509] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1162worker1 # [ 0.449750] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1163worker2 # [ 0.406734] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164lb # [ 0.438325] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1165worker1 # [ 0.449773] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1166client # [ 0.421539] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167worker2 # [ 0.407207] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1168lb # [ 0.438853] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1169worker1 # [ 0.449795] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1170worker2 # [ 0.407394] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1171client # [ 0.421932] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1172lb # [ 0.439137] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1173worker1 # [ 0.449817] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1174lb # [ 0.439155] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1175client # [ 0.422119] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1176worker2 # [ 0.407424] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1177worker1 # [ 0.449838] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1178lb # [ 0.439188] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1179worker2 # [ 0.407881] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180client # [ 0.422401] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1181worker1 # [ 0.449859] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1182worker2 # [ 0.408087] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1183client # [ 0.422589] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1184lb # [ 0.439655] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1185worker1 # [ 0.449881] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1186lb # [ 0.439839] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1187worker1 # [ 0.449902] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1188client # [ 0.422618] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1189worker2 # [ 0.408116] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190lb # [ 0.439855] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1191worker1 # [ 0.449932] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1192client # [ 0.423081] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1193worker2 # [ 0.408589] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1194lb # [ 0.439885] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1195worker1 # [ 0.449942] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1196worker2 # [ 0.408865] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1197client # [ 0.423268] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1198lb # [ 0.440468] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1199worker1 # [ 0.449947] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1200worker2 # [ 0.408883] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1201client # [ 0.423298] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1202worker1 # [ 0.450783] pci 0000:00:07.0: enabling device (0000 -> 0002)1203lb # [ 0.440480] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1204worker2 # [ 0.408913] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1205client # [ 0.423766] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1206lb # [ 0.440485] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1207worker2 # [ 0.409379] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1208client # [ 0.423970] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1209worker2 # [ 0.409563] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1210lb # [ 0.440532] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1211client # [ 0.424001] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1212worker2 # [ 0.409579] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1213lb # [ 0.440591] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1214client # [ 0.424487] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1215worker2 # [ 0.409609] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1216client # [ 0.424764] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1217lb # [ 0.440640] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1218worker2 # [ 0.410191] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1219client # [ 0.424781] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1220lb # [ 0.440695] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1221worker2 # [ 0.410203] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1222client # [ 0.424811] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1223lb # [ 0.440744] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1224worker2 # [ 0.410208] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1225client # [ 0.425287] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1226lb # [ 0.440793] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1227client # [ 0.425472] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1228worker2 # [ 0.410253] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1229client # [ 0.425489] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1230lb # [ 0.440842] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1231worker2 # [ 0.410301] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1232client # [ 0.425518] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1233lb # [ 0.440890] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1234worker2 # [ 0.410348] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1235client # [ 0.426097] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1236worker2 # [ 0.410396] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1237lb # [ 0.440937] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1238client # [ 0.426109] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1239worker2 # [ 0.410443] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1240lb # [ 0.441021] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1241client # [ 0.426114] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1242lb # [ 0.441075] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1243worker2 # [ 0.410490] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1244client # [ 0.426160] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1245lb # [ 0.441097] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1246worker2 # [ 0.410536] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1247client # [ 0.426207] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1248lb # [ 0.441120] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1249worker2 # [ 0.410582] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1250lb # [ 0.441142] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1251client # [ 0.426256] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1252worker2 # [ 0.410628] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1253lb # [ 0.441164] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1254client # [ 0.426303] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1255worker2 # [ 0.410696] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1256lb # [ 0.441186] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1257client # [ 0.426349] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1258worker2 # [ 0.410743] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1259lb # [ 0.441208] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1260client # [ 0.426395] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1261worker2 # [ 0.410764] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1262lb # [ 0.441235] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1263client # [ 0.426441] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1264worker2 # [ 0.410786] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1265lb # [ 0.441256] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1266worker2 # [ 0.410807] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1267lb # [ 0.441278] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1268client # [ 0.426486] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1269worker2 # [ 0.410829] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1270lb # [ 0.441301] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1271client # [ 0.426534] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1272worker2 # [ 0.410850] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1273lb # [ 0.441323] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1274client # [ 0.426606] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1275lb # [ 0.441345] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1276worker2 # [ 0.410871] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1277client # [ 0.426652] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1278lb # [ 0.441367] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1279worker2 # [ 0.410896] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1280client # [ 0.426674] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1281lb # [ 0.441388] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1282worker2 # [ 0.410919] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1283worker1 # [ 0.497563] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1284client # [ 0.426695] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1285lb # [ 0.441410] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1286worker2 # [ 0.410942] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1287worker1 # [ 0.499843] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1288client # [ 0.426717] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1289lb # [ 0.441431] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1290worker2 # [ 0.410965] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1291lb # [ 0.441452] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1292client # [ 0.426738] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1293lb # [ 0.441479] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1294client # [ 0.426760] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1295lb # [ 0.441489] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1296client # [ 0.426781] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1297lb # [ 0.441494] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1298client # [ 0.426806] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1299client # [ 0.426828] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1300client # [ 0.426851] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1301client # [ 0.426874] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1302client # [ 0.426896] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1303client # [ 0.426918] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1304client # [ 0.426939] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1305client # [ 0.426960] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1306client # [ 0.426981] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1307client # [ 0.427002] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1308worker1 # [ 0.510927] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1309client # [ 0.427024] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1310worker2 # [ 0.467035] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1311client # [ 0.427050] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1312worker2 # [ 0.467115] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1313client # [ 0.427060] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1314worker2 # [ 0.467140] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1315client # [ 0.427065] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1316lb # [ 0.486432] pci 0000:00:07.0: enabling device (0000 -> 0002)1317worker2 # [ 0.467163] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1318worker2 # [ 0.467185] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1319worker2 # [ 0.467207] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1320worker2 # [ 0.467230] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1321worker2 # [ 0.467264] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1322worker2 # [ 0.467273] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1323worker2 # [ 0.467278] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1324worker2 # [ 0.468122] pci 0000:00:07.0: enabling device (0000 -> 0002)1325client # [ 0.483956] pci 0000:00:07.0: enabling device (0000 -> 0002)1326worker1 # [ 0.517037] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1327worker1 # [ 0.519235] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1328worker1 # [ 0.529039] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1329worker1 # [ 0.530786] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1330worker1 # [ 0.532573] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1331lb # [ 0.507877] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1332lb # [ 0.510061] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1333worker2 # [ 0.497506] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1334client # [ 0.505584] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1335worker1 # [ 0.543047] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1336lb # [ 0.521931] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1337worker2 # [ 0.507678] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1338client # [ 0.515972] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1339client # [ 0.519796] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1340worker1 # [ 0.553063] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1341worker1 # [ 0.557512] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1342lb # [ 0.531404] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1343client # [ 0.529239] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1344lb # [ 0.535974] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1345lb # [ 0.537970] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1346worker2 # [ 0.519243] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1347worker1 # [ 0.570673] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1348lb # [ 0.539739] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1349worker2 # [ 0.521276] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1350lb # [ 0.541674] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1351worker2 # [ 0.523358] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1352worker2 # [ 0.525533] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1353worker1 # [ 0.573200] msm_serial: driver initialized1354worker1 # [ 0.573337] SuperH (H)SCI(F) driver initialized1355worker1 # [ 0.573403] STM32 USART driver initialized1356worker2 # [ 0.535158] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1357worker2 # [ 0.537274] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1358client # [ 0.540542] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1359client # [ 0.542700] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1360client # [ 0.544522] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1361client # [ 0.546655] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1362lb # [ 0.559613] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1363lb # [ 0.561570] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1364lb # [ 0.565536] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1365worker2 # [ 0.547211] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1366worker2 # [ 0.549559] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1367client # [ 0.556504] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1368client # [ 0.558406] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1369worker2 # [ 0.561147] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1370lb # [ 0.579320] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1371client # [ 0.568569] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1372worker1 # [ 0.607887] loop: module loaded1373worker1 # [ 0.608110] virtio_blk virtio2: 1/0/0 default/read/poll queues1374worker1 # [ 0.610087] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1375lb # [ 0.582134] msm_serial: driver initialized1376lb # [ 0.582295] SuperH (H)SCI(F) driver initialized1377lb # [ 0.582350] STM32 USART driver initialized1378worker2 # [ 0.572243] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1379worker1 # [ 0.613572] megasas: 07.734.00.00-rc11380worker1 # [ 0.614318] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1381client # [ 0.585209] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1382worker2 # [ 0.575120] msm_serial: driver initialized1383worker2 # [ 0.575265] SuperH (H)SCI(F) driver initialized1384worker2 # [ 0.575320] STM32 USART driver initialized1385client # [ 0.587929] msm_serial: driver initialized1386client # [ 0.588070] SuperH (H)SCI(F) driver initialized1387client # [ 0.588123] STM32 USART driver initialized1388worker1 # [ 0.625608] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001389worker1 # [ 0.625640] Intel/Sharp Extended Query Table at 0x00311390worker1 # [ 0.627193] Using buffer write method1391worker1 # [ 0.627261] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1392lb # [ 0.614765] loop: module loaded1393lb # [ 0.614960] virtio_blk virtio2: 1/0/0 default/read/poll queues1394worker1 # [ 0.642612] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001395lb # [ 0.615799] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1396worker1 # [ 0.642648] Intel/Sharp Extended Query Table at 0x00311397worker1 # [ 0.644387] Using buffer write method1398worker1 # [ 0.644418] Concatenating MTD devices:1399worker1 # [ 0.644422] (0): "0.flash"1400worker1 # [ 0.644426] (1): "0.flash"1401worker1 # [ 0.644437] into device "0.flash"1402lb # [ 0.622837] megasas: 07.734.00.00-rc11403lb # [ 0.623594] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1404worker2 # [ 0.609605] loop: module loaded1405worker2 # [ 0.609804] virtio_blk virtio2: 1/0/0 default/read/poll queues1406worker2 # [ 0.610577] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1407client # [ 0.621391] loop: module loaded1408client # [ 0.621587] virtio_blk virtio2: 1/0/0 default/read/poll queues1409client # [ 0.622399] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1410worker2 # [ 0.615634] megasas: 07.734.00.00-rc11411worker2 # [ 0.616360] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1412worker2 # [ 0.618626] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001413worker2 # [ 0.618647] Intel/Sharp Extended Query Table at 0x00311414lb # [ 0.634159] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415client # [ 0.628692] megasas: 07.734.00.00-rc11416lb # [ 0.634201] Intel/Sharp Extended Query Table at 0x00311417client # [ 0.629428] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1418lb # [ 0.635935] Using buffer write method1419lb # [ 0.636029] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1420client # [ 0.631710] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001421client # [ 0.631736] Intel/Sharp Extended Query Table at 0x00311422lb # [ 0.638051] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001423lb # [ 0.638125] Intel/Sharp Extended Query Table at 0x00311424lb # [ 0.639790] Using buffer write method1425lb # [ 0.639832] Concatenating MTD devices:1426worker2 # [ 0.628151] Using buffer write method1427lb # [ 0.639836] (0): "0.flash"1428lb # [ 0.639840] (1): "0.flash"1429lb # [ 0.639851] into device "0.flash"1430worker2 # [ 0.628228] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1431client # [ 0.641012] Using buffer write method1432client # [ 0.641093] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1433worker2 # [ 0.639032] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001434worker2 # [ 0.639057] Intel/Sharp Extended Query Table at 0x00311435worker2 # [ 0.640877] Using buffer write method1436worker2 # [ 0.640905] Concatenating MTD devices:1437worker2 # [ 0.640909] (0): "0.flash"1438worker2 # [ 0.640913] (1): "0.flash"1439worker2 # [ 0.640921] into device "0.flash"1440client # [ 0.651934] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001441client # [ 0.651957] Intel/Sharp Extended Query Table at 0x00311442client # [ 0.653441] Using buffer write method1443client # [ 0.653469] Concatenating MTD devices:1444client # [ 0.653473] (0): "0.flash"1445client # [ 0.653477] (1): "0.flash"1446client # [ 0.653486] into device "0.flash"1447worker1 # [ 0.912982] Freeing initrd memory: 26976K1448worker1 # [ 0.919098] tun: Universal TUN/TAP device driver, 1.61449worker1 # [ 0.922907] thunder_xcv, ver 1.01450worker1 # [ 0.922955] thunder_bgx, ver 1.01451worker1 # [ 0.922978] nicpf, ver 1.01452worker1 # [ 0.923547] e1000: Intel(R) PRO/1000 Network Driver1453worker1 # [ 0.923555] e1000: Copyright (c) 1999-2006 Intel Corporation.1454worker1 # [ 0.923581] e1000e: Intel(R) PRO/1000 Network Driver1455worker1 # [ 0.923589] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1456worker1 # [ 0.923618] igb: Intel(R) Gigabit Ethernet Network Driver1457worker1 # [ 0.923624] igb: Copyright (c) 2007-2014 Intel Corporation.1458worker1 # [ 0.923647] igbvf: Intel(R) Gigabit Virtual Function Network Driver1459worker1 # [ 0.923653] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1460worker1 # [ 0.923791] sky2: driver version 1.301461worker1 # [ 0.932635] usbcore: registered new interface driver usb-storage1462worker1 # [ 0.932745] usbcore: registered new interface driver usbserial_generic1463worker1 # [ 0.932760] usbserial: USB Serial support registered for generic1464worker1 # [ 0.933731] ehci-pci 0000:00:07.0: EHCI Host Controller1465worker1 # [ 0.933757] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11466worker1 # [ 0.934022] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001467worker1 # [ 0.939210] hv_vmbus: registering driver hyperv_keyboard1468worker1 # [ 0.940898] rtc-pl031 9010000.pl031: registered as rtc01469worker1 # [ 0.940923] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T04:43:10 UTC (1790138590)1470worker1 # [ 0.941236] i2c_dev: i2c /dev entries driver1471lb # [ 0.917186] Freeing initrd memory: 26948K1472worker1 # [ 0.946651] sdhci: Secure Digital Host Controller Interface driver1473worker1 # [ 0.946663] sdhci: Copyright(c) Pierre Ossman1474worker1 # [ 0.946944] Synopsys Designware Multimedia Card Interface Driver1475lb # [ 0.923409] tun: Universal TUN/TAP device driver, 1.61476worker1 # [ 0.947326] sdhci-pltfm: SDHCI platform and OF driver helper1477worker1 # [ 0.951599] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001478worker1 # [ 0.951951] hub 1-0:1.0: USB hub found1479worker1 # [ 0.951974] hub 1-0:1.0: 6 ports detected1480worker1 # [ 0.954166] hid: raw HID events driver (C) Jiri Kosina1481worker1 # [ 0.954433] usbcore: registered new interface driver usbhid1482worker1 # [ 0.954439] usbhid: USB HID core driver1483lb # [ 0.927339] thunder_xcv, ver 1.01484lb # [ 0.927390] thunder_bgx, ver 1.01485lb # [ 0.927412] nicpf, ver 1.01486lb # [ 0.928013] e1000: Intel(R) PRO/1000 Network Driver1487worker1 # [ 0.957432] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1488lb # [ 0.928020] e1000: Copyright (c) 1999-2006 Intel Corporation.1489worker2 # [ 0.917070] Freeing initrd memory: 26944K1490worker1 # [ 0.959040] drop_monitor: Initializing network drop monitor service1491lb # [ 0.928045] e1000e: Intel(R) PRO/1000 Network Driver1492worker1 # [ 0.959209] NET: Registered PF_INET6 protocol family1493client # [ 0.924446] Freeing initrd memory: 26972K1494lb # [ 0.928053] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1495worker1 # [ 0.962336] Segment Routing with IPv61496lb # [ 0.928083] igb: Intel(R) Gigabit Ethernet Network Driver1497worker1 # [ 0.962352] In-situ OAM (IOAM) with IPv61498lb # [ 0.928089] igb: Copyright (c) 2007-2014 Intel Corporation.1499worker1 # [ 0.962384] NET: Registered PF_PACKET protocol family1500lb # [ 0.928114] igbvf: Intel(R) Gigabit Virtual Function Network Driver1501lb # [ 0.928120] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1502worker1 # [ 0.964073] 9pnet: Installing 9P2000 support1503lb # [ 0.928263] sky2: driver version 1.301504worker1 # [ 0.964126] Key type dns_resolver registered1505lb # [ 0.929965] usbcore: registered new interface driver usb-storage1506client # [ 0.930623] tun: Universal TUN/TAP device driver, 1.61507lb # [ 0.930062] usbcore: registered new interface driver usbserial_generic1508worker2 # [ 0.923205] tun: Universal TUN/TAP device driver, 1.61509lb # [ 0.930799] ehci-pci 0000:00:07.0: EHCI Host Controller1510lb # [ 0.930833] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11511lb # [ 0.931051] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001512worker2 # [ 0.926944] thunder_xcv, ver 1.01513lb # [ 0.942265] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001514lb # [ 0.942604] hub 1-0:1.0: USB hub found1515lb # [ 0.942620] hub 1-0:1.0: 6 ports detected1516client # [ 0.934395] thunder_xcv, ver 1.01517client # [ 0.934440] thunder_bgx, ver 1.01518client # [ 0.934463] nicpf, ver 1.01519client # [ 0.935017] e1000: Intel(R) PRO/1000 Network Driver1520worker1 # [ 0.970978] registered taskstats version 11521client # [ 0.935024] e1000: Copyright (c) 1999-2006 Intel Corporation.1522worker1 # [ 0.971137] Loading compiled-in X.509 certificates1523lb # [ 0.944701] usbserial: USB Serial support registered for generic1524client # [ 0.935048] e1000e: Intel(R) PRO/1000 Network Driver1525lb # [ 0.945433] hv_vmbus: registering driver hyperv_keyboard1526client # [ 0.935056] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1527worker2 # [ 0.927479] thunder_bgx, ver 1.01528client # [ 0.935085] igb: Intel(R) Gigabit Ethernet Network Driver1529worker2 # [ 0.927509] nicpf, ver 1.01530lb # [ 0.947731] rtc-pl031 9010000.pl031: registered as rtc01531client # [ 0.935091] igb: Copyright (c) 2007-2014 Intel Corporation.1532worker2 # [ 0.928090] e1000: Intel(R) PRO/1000 Network Driver1533client # [ 0.935114] igbvf: Intel(R) Gigabit Virtual Function Network Driver1534worker2 # [ 0.928097] e1000: Copyright (c) 1999-2006 Intel Corporation.1535lb # [ 0.947761] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T04:43:10 UTC (1790138590)1536worker2 # [ 0.928121] e1000e: Intel(R) PRO/1000 Network Driver1537client # [ 0.935120] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1538lb # [ 0.948081] i2c_dev: i2c /dev entries driver1539client # [ 0.935260] sky2: driver version 1.301540worker2 # [ 0.928128] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1541worker2 # [ 0.928156] igb: Intel(R) Gigabit Ethernet Network Driver1542worker2 # [ 0.928161] igb: Copyright (c) 2007-2014 Intel Corporation.1543worker2 # [ 0.928183] igbvf: Intel(R) Gigabit Virtual Function Network Driver1544worker1 # [ 0.980118] Demotion targets for Node 0: null1545client # [ 0.944567] ehci-pci 0000:00:07.0: EHCI Host Controller1546worker1 # [ 0.980234] Key type .fscrypt registered1547worker2 # [ 0.928189] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1548client # [ 0.944600] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11549worker1 # [ 0.980241] Key type fscrypt-provisioning registered1550worker2 # [ 0.928326] sky2: driver version 1.301551lb # [ 0.953725] sdhci: Secure Digital Host Controller Interface driver1552lb # [ 0.953741] sdhci: Copyright(c) Pierre Ossman1553client # [ 0.944763] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001554worker1 # [ 0.980337] ima: No TPM chip found, activating TPM-bypass!1555worker2 # [ 0.929918] usbcore: registered new interface driver usb-storage1556lb # [ 0.954013] Synopsys Designware Multimedia Card Interface Driver1557worker1 # [ 0.980357] ima: Allocated hash algorithm: sha11558worker2 # [ 0.929968] usbcore: registered new interface driver usbserial_generic1559worker1 # [ 0.980377] ima: No architecture policies found1560client # [ 0.947286] usbcore: registered new interface driver usb-storage1561worker2 # [ 0.929982] usbserial: USB Serial support registered for generic1562worker2 # [ 0.930578] hv_vmbus: registering driver hyperv_keyboard1563client # [ 0.947387] usbcore: registered new interface driver usbserial_generic1564worker1 # [ 0.984440] input: gpio-keys as /devices/platform/gpio-keys/input/input01565worker2 # [ 0.931729] ehci-pci 0000:00:07.0: EHCI Host Controller1566client # [ 0.947401] usbserial: USB Serial support registered for generic1567lb # [ 0.956393] sdhci-pltfm: SDHCI platform and OF driver helper1568worker2 # [ 0.931756] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11569lb # [ 0.957996] hid: raw HID events driver (C) Jiri Kosina1570client # [ 0.950333] hv_vmbus: registering driver hyperv_keyboard1571worker2 # [ 0.932036] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001572lb # [ 0.959600] usbcore: registered new interface driver usbhid1573worker2 # [ 0.943946] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001574lb # [ 0.959611] usbhid: USB HID core driver1575worker2 # [ 0.944259] hub 1-0:1.0: USB hub found1576client # [ 0.951907] rtc-pl031 9010000.pl031: registered as rtc01577worker2 # [ 0.944279] hub 1-0:1.0: 6 ports detected1578lb # [ 0.961981] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1579client # [ 0.951935] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T04:43:10 UTC (1790138590)1580client # [ 0.952252] i2c_dev: i2c /dev entries driver1581worker2 # [ 0.946782] rtc-pl031 9010000.pl031: registered as rtc01582worker2 # [ 0.946814] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T04:43:10 UTC (1790138590)1583lb # [ 0.964697] drop_monitor: Initializing network drop monitor service1584worker2 # [ 0.948926] i2c_dev: i2c /dev entries driver1585lb # [ 0.964854] NET: Registered PF_INET6 protocol family1586lb # [ 0.967097] Segment Routing with IPv61587lb # [ 0.967120] In-situ OAM (IOAM) with IPv61588lb # [ 0.967150] NET: Registered PF_PACKET protocol family1589client # [ 0.957845] sdhci: Secure Digital Host Controller Interface driver1590client # [ 0.957862] sdhci: Copyright(c) Pierre Ossman1591lb # [ 0.968825] 9pnet: Installing 9P2000 support1592client # [ 0.958139] Synopsys Designware Multimedia Card Interface Driver1593lb # [ 0.968877] Key type dns_resolver registered1594client # [ 0.958519] sdhci-pltfm: SDHCI platform and OF driver helper1595worker2 # [ 0.952701] sdhci: Secure Digital Host Controller Interface driver1596worker2 # [ 0.952718] sdhci: Copyright(c) Pierre Ossman1597client # [ 0.959953] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001598client # [ 0.960344] hub 1-0:1.0: USB hub found1599worker2 # [ 0.952980] Synopsys Designware Multimedia Card Interface Driver1600client # [ 0.960365] hub 1-0:1.0: 6 ports detected1601worker2 # [ 0.953354] sdhci-pltfm: SDHCI platform and OF driver helper1602worker2 # [ 0.954904] hid: raw HID events driver (C) Jiri Kosina1603worker2 # [ 0.958587] usbcore: registered new interface driver usbhid1604client # [ 0.965576] hid: raw HID events driver (C) Jiri Kosina1605worker2 # [ 0.958597] usbhid: USB HID core driver1606client # [ 0.965826] usbcore: registered new interface driver usbhid1607client # [ 0.965835] usbhid: USB HID core driver1608worker1 # [ 1.002218] clk: Disabling unused clocks1609worker1 # [ 1.002250] PM: genpd: Disabling unused power domains1610lb # [ 0.975806] registered taskstats version 11611lb # [ 0.975946] Loading compiled-in X.509 certificates1612worker2 # [ 0.960987] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1613client # [ 0.968849] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1614worker2 # [ 0.962541] drop_monitor: Initializing network drop monitor service1615client # [ 0.970424] drop_monitor: Initializing network drop monitor service1616worker1 # [ 1.006476] Freeing unused kernel memory: 4736K1617worker1 # [ 1.006666] Run /init as init process1618worker2 # [ 0.962689] NET: Registered PF_INET6 protocol family1619client # [ 0.970589] NET: Registered PF_INET6 protocol family1620worker2 # [ 0.965770] Segment Routing with IPv61621client # [ 0.973667] Segment Routing with IPv61622worker2 # [ 0.965790] In-situ OAM (IOAM) with IPv61623client # [ 0.973686] In-situ OAM (IOAM) with IPv61624worker2 # [ 0.965819] NET: Registered PF_PACKET protocol family1625client # [ 0.973714] NET: Registered PF_PACKET protocol family1626client # [ 0.975401] 9pnet: Installing 9P2000 support1627client # [ 0.975452] Key type dns_resolver registered1628lb # [ 0.984818] Demotion targets for Node 0: null1629lb # [ 0.984933] Key type .fscrypt registered1630lb # [ 0.984940] Key type fscrypt-provisioning registered1631lb # [ 0.985042] ima: No TPM chip found, activating TPM-bypass!1632worker2 # [ 0.967741] 9pnet: Installing 9P2000 support1633lb # [ 0.985062] ima: Allocated hash algorithm: sha11634worker2 # [ 0.967789] Key type dns_resolver registered1635lb # [ 0.985085] ima: No architecture policies found1636lb # [ 0.989184] input: gpio-keys as /devices/platform/gpio-keys/input/input01637worker2 # [ 0.974513] registered taskstats version 11638worker2 # [ 0.974659] Loading compiled-in X.509 certificates1639client # [ 0.982402] registered taskstats version 11640client # [ 0.982549] Loading compiled-in X.509 certificates1641worker1 # [ 1.023210] systemd[1]: Successfully made /usr/ read-only.1642worker2 # [ 0.983449] Demotion targets for Node 0: null1643client # [ 0.991448] Demotion targets for Node 0: null1644worker2 # [ 0.983565] Key type .fscrypt registered1645client # [ 0.991557] Key type .fscrypt registered1646worker2 # [ 0.983572] Key type fscrypt-provisioning registered1647client # [ 0.991564] Key type fscrypt-provisioning registered1648worker2 # [ 0.983670] ima: No TPM chip found, activating TPM-bypass!1649client # [ 0.991664] ima: No TPM chip found, activating TPM-bypass!1650worker2 # [ 0.983690] ima: Allocated hash algorithm: sha11651client # [ 0.991684] ima: Allocated hash algorithm: sha11652worker2 # [ 0.983711] ima: No architecture policies found1653client # [ 0.991707] ima: No architecture policies found1654client # [ 0.995832] input: gpio-keys as /devices/platform/gpio-keys/input/input01655worker2 # [ 0.988045] input: gpio-keys as /devices/platform/gpio-keys/input/input01656lb # [ 1.007295] clk: Disabling unused clocks1657lb # [ 1.007329] PM: genpd: Disabling unused power domains1658lb # [ 1.011550] Freeing unused kernel memory: 4736K1659lb # [ 1.011765] Run /init as init process1660worker2 # [ 1.006580] clk: Disabling unused clocks1661worker2 # [ 1.006611] PM: genpd: Disabling unused power domains1662client # [ 1.014651] clk: Disabling unused clocks1663client # [ 1.014679] PM: genpd: Disabling unused power domains1664worker2 # [ 1.010802] Freeing unused kernel memory: 4736K1665client # [ 1.018920] Freeing unused kernel memory: 4736K1666worker2 # [ 1.011609] Run /init as init process1667client # [ 1.019121] Run /init as init process1668lb # [ 1.028521] systemd[1]: Successfully made /usr/ read-only.1669worker2 # [ 1.025340] systemd[1]: Successfully made /usr/ read-only.1670client # [ 1.034246] systemd[1]: Successfully made /usr/ read-only.1671worker1 # [ 1.196974] usb 1-1: new high-speed USB device number 2 using ehci-pci1672lb # [ 1.190115] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker2 # [ 1.191076] usb 1-1: new high-speed USB device number 2 using ehci-pci1674client # [ 1.208007] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker1 # [ 1.349451] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker1 # [ 1.358028] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1677worker1 # [ 1.370753] systemd[1]: Detected virtualization qemu.1678worker1 # [ 1.372952] systemd[1]: Detected architecture arm64.1679lb # [ 1.342685] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11680worker1 # [ 1.374901] systemd[1]: Running in initrd.1681worker1 # [ 1.377653] systemd[1]: Initializing machine ID from random generator.1682worker1 # [ 1.380700] systemd[1]: Hostname set to <worker1>.1683worker2 # [ 1.341109] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684lb # [ 1.363255] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1685lb # [ 1.367634] systemd[1]: Detected virtualization qemu.1686lb # [ 1.368392] systemd[1]: Detected architecture arm64.1687client # [ 1.358019] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11688lb # [ 1.369040] systemd[1]: Running in initrd.1689lb # [ 1.370004] systemd[1]: Initializing machine ID from random generator.1690lb # [ 1.371092] systemd[1]: Hostname set to <lb>.1691worker2 # [ 1.360228] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692client # [ 1.368875] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1693worker2 # [ 1.372677] systemd[1]: Detected virtualization qemu.1694client # [ 1.368904] systemd[1]: Detected virtualization qemu.1695worker2 # [ 1.374686] systemd[1]: Detected architecture arm64.1696client # [ 1.368938] systemd[1]: Detected architecture arm64.1697worker2 # [ 1.376760] systemd[1]: Running in initrd.1698client # [ 1.368946] systemd[1]: Running in initrd.1699client # [ 1.369395] systemd[1]: Initializing machine ID from random generator.1700worker2 # [ 1.379498] systemd[1]: Initializing machine ID from random generator.1701client # [ 1.369509] systemd[1]: Hostname set to <client>.1702worker2 # [ 1.382427] systemd[1]: Hostname set to <worker2>.1703worker1 # [ 1.441132] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704lb # [ 1.426339] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705client # [ 1.436105] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker2 # [ 1.427289] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker1 # [ 1.560960] usb 1-2: new high-speed USB device number 3 using ehci-pci1708lb # [ 1.546201] usb 1-2: new high-speed USB device number 3 using ehci-pci1709client # [ 1.555950] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker2 # [ 1.555035] usb 1-2: new high-speed USB device number 3 using ehci-pci1711lb # [ 1.629345] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712worker1 # [ 1.699230] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713client # [ 1.672736] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714worker2 # [ 1.679901] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker1 # [ 1.719430] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21716lb # [ 1.701262] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21717lb # [ 1.703982] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01718worker1 # [ 1.729251] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01719lb # [ 1.716588] systemd[1]: Queued start job for default target Initrd Default Target.1720client # [ 1.710341] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721client # [ 1.716338] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722worker2 # [ 1.710215] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21723lb # [ 1.728803] systemd[1]: Created slice Slice /system/modprobe.1724worker2 # [ 1.713139] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01725lb # [ 1.730198] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1726lb # [ 1.730245] systemd[1]: Expecting device /dev/disk/by-label/nixos...1727lb # [ 1.730275] systemd[1]: Reached target Path Units.1728lb # [ 1.730298] systemd[1]: Reached target Slice Units.1729lb # [ 1.730319] systemd[1]: Reached target Swaps.1730lb # [ 1.730341] systemd[1]: Reached target Timer Units.1731lb # [ 1.730557] systemd[1]: Listening on D-Bus System Message Bus Socket.1732lb # [ 1.730732] systemd[1]: Listening on Journal Socket (/dev/log).1733lb # [ 1.730895] systemd[1]: Listening on Journal Sockets.1734lb # [ 1.731043] systemd[1]: Listening on udev Control Socket.1735lb # [ 1.731165] systemd[1]: Listening on udev Kernel Socket.1736lb # [ 1.731187] systemd[1]: Reached target Socket Units.1737lb # [ 1.744404] systemd[1]: Starting Create List of Static Device Nodes...1738lb # [ 1.745655] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1739lb # [ 1.757532] systemd[1]: Mounting Kernel Configuration File System...1740worker2 # [ 1.763586] systemd[1]: Queued start job for default target Initrd Default Target.1741client # [ 1.776961] systemd[1]: Queued start job for default target Initrd Default Target.1742lb # [ 1.786371] systemd[1]: Starting Journal Service...1743lb # [ 1.789024] systemd[1]: Starting Load Kernel Modules...1744worker2 # [ 1.773332] systemd[1]: Created slice Slice /system/modprobe.1745worker1 # [ 1.816795] systemd[1]: Queued start job for default target Initrd Default Target.1746lb # [ 1.789941] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1747worker2 # [ 1.774433] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1748worker2 # [ 1.775706] systemd[1]: Expecting device /dev/disk/by-label/nixos...1749worker2 # [ 1.776682] systemd[1]: Reached target Path Units.1750worker2 # [ 1.777413] systemd[1]: Reached target Slice Units.1751worker2 # [ 1.778155] systemd[1]: Reached target Swaps.1752client # [ 1.786289] systemd[1]: Created slice Slice /system/modprobe.1753worker2 # [ 1.778842] systemd[1]: Reached target Timer Units.1754worker2 # [ 1.779825] systemd[1]: Listening on D-Bus System Message Bus Socket.1755client # [ 1.787496] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1756worker2 # [ 1.780966] systemd[1]: Listening on Journal Socket (/dev/log).1757client # [ 1.788960] systemd[1]: Expecting device /dev/disk/by-label/nixos...1758worker2 # [ 1.781995] systemd[1]: Listening on Journal Sockets.1759client # [ 1.790021] systemd[1]: Reached target Path Units.1760client # [ 1.790805] systemd[1]: Reached target Slice Units.1761worker2 # [ 1.782897] systemd[1]: Listening on udev Control Socket.1762client # [ 1.791632] systemd[1]: Reached target Swaps.1763lb # [ 1.800046] systemd[1]: Starting Coldplug All udev Devices...1764worker2 # [ 1.784019] systemd[1]: Listening on udev Kernel Socket.1765client # [ 1.792433] systemd[1]: Reached target Timer Units.1766worker2 # [ 1.784840] systemd[1]: Reached target Socket Units.1767worker1 # [ 1.828681] systemd[1]: Created slice Slice /system/modprobe.1768client # [ 1.793433] systemd[1]: Listening on D-Bus System Message Bus Socket.1769worker1 # [ 1.829991] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1770client # [ 1.794781] systemd[1]: Listening on Journal Socket (/dev/log).1771worker2 # [ 1.787142] systemd[1]: Starting Create List of Static Device Nodes...1772worker1 # [ 1.831328] systemd[1]: Expecting device /dev/disk/by-label/nixos...1773client # [ 1.795932] systemd[1]: Listening on Journal Sockets.1774worker2 # [ 1.788204] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1775worker1 # [ 1.832433] systemd[1]: Reached target Path Units.1776client # [ 1.796932] systemd[1]: Listening on udev Control Socket.1777worker1 # [ 1.833351] systemd[1]: Reached target Slice Units.1778client # [ 1.797940] systemd[1]: Listening on udev Kernel Socket.1779worker1 # [ 1.834189] systemd[1]: Reached target Swaps.1780client # [ 1.798835] systemd[1]: Reached target Socket Units.1781worker1 # [ 1.834943] systemd[1]: Reached target Timer Units.1782worker1 # [ 1.835989] systemd[1]: Listening on D-Bus System Message Bus Socket.1783client # [ 1.801382] systemd[1]: Starting Create List of Static Device Nodes...1784client # [ 1.802542] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1785worker1 # [ 1.837253] systemd[1]: Listening on Journal Socket (/dev/log).1786worker1 # [ 1.837408] systemd[1]: Listening on Journal Sockets.1787worker1 # [ 1.837548] systemd[1]: Listening on udev Control Socket.1788worker2 # [ 1.797869] systemd[1]: Mounting Kernel Configuration File System...1789worker1 # [ 1.837665] systemd[1]: Listening on udev Kernel Socket.1790worker1 # [ 1.837689] systemd[1]: Reached target Socket Units.1791worker1 # [ 1.843961] systemd[1]: Starting Create List of Static Device Nodes...1792worker1 # [ 1.844051] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1793lb # [ 1.818248] systemd[1]: Finished Create List of Static Device Nodes.1794client # [ 1.812165] systemd[1]: Mounting Kernel Configuration File System...1795lb # [ 1.819052] systemd[1]: Mounted Kernel Configuration File System.1796worker1 # [ 1.849873] systemd[1]: Mounting Kernel Configuration File System...1797client # [ 1.820188] systemd[1]: Starting Journal Service...1798lb # [ 1.830615] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1799worker2 # [ 1.815231] systemd[1]: Starting Journal Service...1800worker1 # [ 1.865064] systemd[1]: Starting Journal Service...1801worker2 # [ 1.826921] systemd[1]: Starting Load Kernel Modules...1802worker2 # [ 1.827988] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1803worker1 # [ 1.877330] systemd[1]: Starting Load Kernel Modules...1804worker1 # [ 1.877466] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1805client # [ 1.844165] systemd[1]: Starting Load Kernel Modules...1806client # [ 1.844331] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1807lb # [ 1.857054] systemd-journald[72]: Collecting audit messages is disabled.1808worker2 # [ 1.843453] systemd[1]: Starting Coldplug All udev Devices...1809client # [ 1.866817] systemd-journald[72]: Collecting audit messages is disabled.1810worker2 # [ 1.859326] systemd[1]: Finished Create List of Static Device Nodes.1811worker2 # [ 1.860124] systemd[1]: Mounted Kernel Configuration File System.1812lb # [ 1.876882] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1813client # [ 1.870324] systemd[1]: Starting Coldplug All udev Devices...1814worker1 # [ 1.907389] systemd-journald[72]: Collecting audit messages is disabled.1815worker1 # [ 1.908585] systemd[1]: Starting Coldplug All udev Devices...1816client # [ 1.884097] systemd[1]: Finished Create List of Static Device Nodes.1817client # [ 1.884880] systemd[1]: Mounted Kernel Configuration File System.1818lb # [ 1.898154] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1819worker2 # [ 1.880407] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1820worker1 # [ 1.925039] systemd[1]: Finished Create List of Static Device Nodes.1821worker1 # [ 1.925742] systemd[1]: Mounted Kernel Configuration File System.1822lb # [ 1.901195] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1823lb # [ 1.903925] systemd[1]: Starting Create Static Device Nodes in /dev...1824worker2 # [ 1.888646] systemd-journald[72]: Collecting audit messages is disabled.1825lb # [ 1.915479] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01826client # [ 1.908088] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1827lb # [ 1.915729] [drm] features: -virgl +edid -resource_blob -host_visible1828worker1 # [ 1.937679] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1829lb # [ 1.915739] [drm] features: -context_init1830lb # [ 1.916504] [drm] number of scanouts: 11831lb # [ 1.916522] [drm] number of cap sets: 01832worker2 # [ 1.902370] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1833worker2 # [ 1.915050] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1834worker1 # [ 1.965732] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1835worker2 # [ 1.924305] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01836worker2 # [ 1.924561] [drm] features: -virgl +edid -resource_blob -host_visible1837lb # [ 1.934469] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1838worker2 # [ 1.924571] [drm] features: -context_init1839worker2 # [ 1.925287] [drm] number of scanouts: 11840lb # [ 1.934491] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01841worker2 # [ 1.925305] [drm] number of cap sets: 01842worker1 # [ 1.980464] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1843client # [ 1.944664] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1844client # [ 1.947261] systemd[1]: Starting Create Static Device Nodes in /dev...1845worker1 # [ 1.983641] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1846worker1 # [ 1.986301] systemd[1]: Starting Create Static Device Nodes in /dev...1847worker2 # [ 1.943737] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1848worker2 # [ 1.947388] systemd[1]: Starting Create Static Device Nodes in /dev...1849worker1 # [ 1.994446] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01850worker1 # [ 1.994726] [drm] features: -virgl +edid -resource_blob -host_visible1851lb # [ 1.962393] Console: switching to colour frame buffer device 160x501852worker1 # [ 1.994736] [drm] features: -context_init1853worker1 # [ 1.995483] [drm] number of scanouts: 11854worker1 # [ 1.995501] [drm] number of cap sets: 01855client # [ 1.966710] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1856worker2 # [ 1.955371] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1857worker2 # [ 1.955392] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01858lb # [ 1.978784] systemd[1]: Finished Create Static Device Nodes in /dev.1859lb # [ 1.979173] systemd[1]: Reached target Preparation for Local File Systems.1860lb # [ 1.979198] systemd[1]: Reached target Local File Systems.1861lb # [ 1.986488] systemd[1]: Starting Rule-based Manager for Device Events and Files...1862client # [ 1.984048] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1863lb # [ 1.994812] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1864worker2 # [ 1.975318] Console: switching to colour frame buffer device 160x501865worker1 # [ 2.021283] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1866worker1 # [ 2.021304] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01867client # [ 1.997229] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01868client # [ 1.997482] [drm] features: -virgl +edid -resource_blob -host_visible1869client # [ 1.997493] [drm] features: -context_init1870client # [ 1.998224] [drm] number of scanouts: 11871lb # [ 2.009137] systemd[1]: Finished Load Kernel Modules.1872client # [ 1.998243] [drm] number of cap sets: 01873lb # [ 2.014218] systemd[1]: Starting Apply Kernel Variables...1874client # [ 2.012041] systemd[1]: Finished Create Static Device Nodes in /dev.1875client # [ 2.012480] systemd[1]: Reached target Preparation for Local File Systems.1876client # [ 2.012512] systemd[1]: Reached target Local File Systems.1877worker2 # [ 2.007743] systemd[1]: Finished Create Static Device Nodes in /dev.1878worker2 # [ 2.008133] systemd[1]: Reached target Preparation for Local File Systems.1879worker2 # [ 2.008160] systemd[1]: Reached target Local File Systems.1880worker2 # [ 2.012526] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1881worker1 # [ 2.049249] Console: switching to colour frame buffer device 160x501882worker1 # [ 2.056116] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1883client # [ 2.020435] systemd[1]: Starting Rule-based Manager for Device Events and Files...1884worker2 # [ 2.013942] systemd[1]: Starting Rule-based Manager for Device Events and Files...1885client # [ 2.025424] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1886client # [ 2.025449] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01887worker1 # [ 2.069582] systemd[1]: Finished Create Static Device Nodes in /dev.1888worker1 # [ 2.069988] systemd[1]: Reached target Preparation for Local File Systems.1889worker1 # [ 2.070026] systemd[1]: Reached target Local File Systems.1890worker1 # [ 2.075196] systemd[1]: Starting Rule-based Manager for Device Events and Files...1891lb # [ 2.040381] systemd-modules-load[73]: Inserted module 'dm_mod'1892lb # [ 2.055664] systemd[1]: Started Journal Service.1893lb # [ 2.044846] systemd-modules-load[73]: Module 'virtio_balloon' is built in1894lb # [ 2.046000] systemd-modules-load[73]: Module 'virtio_console' is built in1895worker1 # [ 2.070782] systemd-modules-load[74]: Inserted module 'dm_mod'1896lb # [ 2.047098] systemd-modules-load[73]: Inserted module 'virtio_gpu'1897worker1 # [ 2.088864] systemd[1]: Started Journal Service.1898worker2 # [ 2.043255] systemd[1]: Finished Load Kernel Modules.1899client # [ 2.046000] Console: switching to colour frame buffer device 160x501900worker2 # [ 2.046754] systemd[1]: Starting Apply Kernel Variables...1901worker1 # [ 2.074892] systemd-modules-load[74]: Module 'virtio_balloon' is built in1902worker1 # [ 2.081252] systemd-modules-load[74]: Module 'virtio_console' is built in1903worker1 # [ 2.083265] systemd-modules-load[74]: Inserted module 'virtio_gpu'1904client # [ 2.048703] systemd-modules-load[74]: Inserted module 'dm_mod'1905worker1 # [ 2.084360] systemd-modules-load[74]: Module 'virtio_rng' is built in1906client # [ 2.050459] systemd-modules-load[74]: Module 'virtio_balloon' is built in1907client # [ 2.051594] systemd-modules-load[74]: Module 'virtio_console' is built in1908client # [ 2.068645] systemd[1]: Started Journal Service.1909lb # [ 2.063913] systemd-modules-load[73]: Module 'virtio_rng' is built in1910worker1 # [ 2.093635] systemd[1]: Finished Load Kernel Modules.1911lb # [ 2.069913] systemd[1]: Starting Create System Files and Directories...1912client # [ 2.061138] systemd[1]: Starting Create System Files and Directories...1913client # [ 2.082091] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1914worker1 # [ 2.103808] systemd[1]: Starting Apply Kernel Variables...1915worker2 # [ 2.064636] systemd-modules-load[73]: Inserted module 'dm_mod'1916lb # [ 2.084098] systemd[1]: Finished Apply Kernel Variables.1917worker2 # [ 2.080810] systemd[1]: Started Journal Service.1918lb # [ 2.086302] systemd-udevd[78]: Using default interface naming scheme 'v261'.1919worker2 # [ 2.069392] systemd-modules-load[73]: Module 'virtio_balloon' is built in1920worker2 # [ 2.070572] systemd-modules-load[73]: Module 'virtio_console' is built in1921client # [ 2.077897] systemd-modules-load[74]: Inserted module 'virtio_gpu'1922worker2 # [ 2.071635] systemd-modules-load[73]: Inserted module 'virtio_gpu'1923client # [ 2.080370] systemd-modules-load[74]: Module 'virtio_rng' is built in1924client # [ 2.093162] systemd[1]: Finished Load Kernel Modules.1925worker2 # [ 2.088515] systemd-modules-load[73]: Module 'virtio_rng' is built in1926worker2 # [ 2.089665] systemd[1]: Starting Create System Files and Directories...1927client # [ 2.097804] systemd[1]: Starting Apply Kernel Variables...1928lb # [ 2.108274] systemd[1]: Finished Create System Files and Directories.1929worker1 # [ 2.136094] systemd[1]: Starting Create System Files and Directories...1930worker2 # [ 2.107125] systemd[1]: Finished Apply Kernel Variables.1931lb # [ 2.126776] systemd[1]: Started Rule-based Manager for Device Events and Files.1932worker1 # [ 2.157404] systemd-udevd[78]: Using default interface naming scheme 'v261'.1933client # [ 2.125388] systemd-udevd[78]: Using default interface naming scheme 'v261'.1934worker2 # [ 2.120391] systemd-udevd[79]: Using default interface naming scheme 'v261'.1935worker2 # [ 2.129365] systemd[1]: Finished Create System Files and Directories.1936worker1 # [ 2.172881] systemd[1]: Finished Create System Files and Directories.1937worker1 # [ 2.179690] systemd[1]: Finished Apply Kernel Variables.1938client # [ 2.157870] systemd[1]: Finished Create System Files and Directories.1939worker2 # [ 2.153329] systemd[1]: Started Rule-based Manager for Device Events and Files.1940worker1 # [ 2.201657] systemd[1]: Started Rule-based Manager for Device Events and Files.1941client # [ 2.167830] systemd[1]: Finished Apply Kernel Variables.1942lb # [ 2.192133] systemd[1]: Starting Virtual Console Setup...1943client # [ 2.187631] systemd[1]: Started Rule-based Manager for Device Events and Files.1944worker2 # [ 2.212108] systemd[1]: Starting Virtual Console Setup...1945worker1 # [ 2.260197] systemd[1]: Starting Virtual Console Setup...1946lb # [ 2.240525] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1947lb # [ 2.244076] systemd[1]: Finished Virtual Console Setup.1948client # [ 2.252144] systemd[1]: Starting Virtual Console Setup...1949worker2 # [ 2.265309] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1950worker1 # [ 2.308522] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1951worker2 # [ 2.268096] systemd[1]: Finished Virtual Console Setup.1952worker1 # [ 2.312108] systemd[1]: Finished Virtual Console Setup.1953client # [ 2.296522] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954client # [ 2.300234] systemd[1]: Finished Virtual Console Setup.1955lb # [ 2.851541] systemd[1]: Finished Coldplug All udev Devices.1956lb # [ 2.852641] systemd[1]: Reached target System Initialization.1957lb # [ 2.853444] systemd[1]: Reached target Basic System.1958worker2 # [ 2.869349] systemd[1]: Finished Coldplug All udev Devices.1959worker2 # [ 2.870393] systemd[1]: Reached target System Initialization.1960worker2 # [ 2.871210] systemd[1]: Reached target Basic System.1961worker1 # [ 2.913110] systemd[1]: Finished Coldplug All udev Devices.1962worker1 # [ 2.914111] systemd[1]: Reached target System Initialization.1963worker1 # [ 2.914924] systemd[1]: Reached target Basic System.1964client # [ 2.908767] systemd[1]: Finished Coldplug All udev Devices.1965client # [ 2.910254] systemd[1]: Reached target System Initialization.1966client # [ 2.911282] systemd[1]: Reached target Basic System.1967lb # [ 2.993552] (udev-worker)[103]: Network interface NamePolicy= disabled on kernel command line.1968lb # [ 3.006273] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969lb # [ 3.012297] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line.1970worker2 # [ 3.016219] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971worker2 # [ 3.040467] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1972worker1 # [ 3.087076] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1973worker2 # [ 3.052383] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1974worker1 # [ 3.109586] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975client # [ 3.076411] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1976worker1 # [ 3.115953] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line.1977lb # [ 3.093365] systemd[1]: Found device /dev/disk/by-label/nixos.1978lb # [ 3.099332] systemd[1]: Reached target Initrd Root Device.1979client # [ 3.088074] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1980client # [ 3.091076] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1981lb # [ 3.103215] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982worker2 # [ 3.112868] systemd[1]: Found device /dev/disk/by-label/nixos.1983worker2 # [ 3.115955] systemd[1]: Reached target Initrd Root Device.1984worker1 # [ 3.159689] systemd[1]: Found device /dev/disk/by-label/nixos.1985worker2 # [ 3.118747] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986worker1 # [ 3.162442] systemd[1]: Reached target Initrd Root Device.1987worker1 # [ 3.165132] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1988lb # [ 3.159251] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1989lb # [ 3.166978] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1990lb # [ 3.169483] systemd[1]: Mounting /sysroot...1991client # [ 3.176511] systemd[1]: Found device /dev/disk/by-label/nixos.1992worker2 # [ 3.169656] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1993client # [ 3.177649] systemd[1]: Reached target Initrd Root Device.1994worker1 # [ 3.215593] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1995client # [ 3.181251] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1996worker2 # [ 3.176130] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1997worker1 # [ 3.224088] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998worker2 # [ 3.185323] systemd[1]: Mounting /sysroot...1999worker1 # [ 3.230804] systemd[1]: Mounting /sysroot...2000lb # [ 3.226662] EXT4-fs (vda): mounted filesystem 779349b4-bc2a-4469-83b4-5756cd0f9b62 r/w with ordered data mode. Quota mode: none.
loading 2,436 hidden lines…
4437client # farm-top> LOG-farm-top-t1b4438worker2 # [ 55.624697] nix-daemon[859]: reaped child process 1013, status = succeeded4439worker2 # [ 55.711352] nix-daemon[859]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4440worker2 # [ 55.732305] nix-daemon[859]: reaped child process 1032, status = succeeded4441lb # [ 55.755682] niks3-server[1028]: 2026/09/23 04:44:05 INFO Received uploads request method=POST path=/api/pending_closures4442worker2 # [ 55.752473] nix-grpc-daemon[874]: time=2026-09-23T04:44:05.318Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4443worker2 # [ 55.754343] nix-grpc-daemon[874]: time=2026-09-23T04:44:05.318Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4444lb # [ 55.808410] niks3-server[1028]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4445worker2 # [ 55.810484] nix-grpc-daemon[874]: time=2026-09-23T04:44:05.378Z level=INFO msg="Uploading 1 narinfos"4446lb # [ 55.821080] niks3-server[1028]: 2026/09/23 04:44:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4447lb # [ 55.832145] niks3-server[1028]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4448lb # [ 55.840188] niks3-server[1028]: 2026/09/23 04:44:05 INFO Signed narinfos id=8 count=14449lb # [ 55.842997] niks3-server[1028]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4450lb # [ 55.851911] niks3-server[1028]: 2026/09/23 04:44:05 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4451lb # [ 55.861235] niks3-server[1028]: 2026/09/23 04:44:05 INFO Completed upload id=84452worker2 # [ 55.844605] nix-grpc-daemon[874]: time=2026-09-23T04:44:05.412Z level=INFO msg="Upload complete. (309ms)"4453lb # [ 55.864828] niks3-server[1028]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4454worker2 # [ 55.861214] nix-grpc-daemon[771]: ts=2026-09-23T04:44:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14455worker2 # [ 55.880135] nix-daemon[859]: reaped child process 1011, status = succeeded4456client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.22 seconds)4457client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4458client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4459lb: must succeed: curl -sf localhost:9901/clusters4460lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4461lb: must succeed: curl -sf localhost:9901/clusters4462lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4463worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4464worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4465(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.76 seconds)4466subtest: outputs read back through any worker carry the cache signature4467client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24468client # don't know how to build these paths:4469client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4470worker1 # [ 56.509744] nix-grpc-daemon[767]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:385504471worker1 # [ 56.516869] nix-grpc-daemon[767]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385504472worker2 # [ 56.500150] nix-grpc-daemon[771]: ts=2026-09-23T04:44:06Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04473worker2 # [ 56.506736] nix-grpc-daemon[771]: ts=2026-09-23T04:44:06Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=14474worker1 # [ 56.551741] nix-grpc-daemon[767]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504475worker1 # [ 56.567974] nix-grpc-daemon[767]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385504476client # copying 1 paths to 'lb:50051'...4477client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4478worker1 # [ 56.584503] nix-grpc-daemon[767]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385504479worker1 # [ 56.600149] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4480worker1 # [ 56.630588] nix-grpc-daemon[767]: ts=2026-09-23T04:44:06Z level=info event=niks3_push_started pid=8614481worker1 # [ 56.803320] nix-grpc-daemon[861]: time=2026-09-23T04:44:06.376Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4482worker1 # [ 56.918006] nix-daemon[835]: accepted connection from pid 874, user nix-grpc-daemon (trusted)4483worker1 # [ 56.941657] nix-daemon[835]: reaped child process 877, status = succeeded4484lb # [ 56.921519] niks3-server[1028]: 2026/09/23 04:44:06 INFO Received uploads request method=POST path=/api/pending_closures4485worker1 # [ 56.955911] nix-grpc-daemon[861]: time=2026-09-23T04:44:06.530Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4486worker1 # [ 56.960003] nix-grpc-daemon[861]: time=2026-09-23T04:44:06.530Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4487lb # [ 56.959884] niks3-server[1028]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4488worker1 # [ 56.999319] nix-grpc-daemon[861]: time=2026-09-23T04:44:06.574Z level=INFO msg="Uploading 1 narinfos"4489lb # [ 56.973546] niks3-server[1028]: 2026/09/23 04:44:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4490lb # [ 56.984318] niks3-server[1028]: 2026/09/23 04:44:06 INFO Signed narinfos id=9 count=14491lb # [ 56.990536] niks3-server[1028]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4492lb # [ 57.005847] niks3-server[1028]: 2026/09/23 04:44:06 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4493worker1 # [ 57.040388] nix-grpc-daemon[861]: time=2026-09-23T04:44:06.616Z level=INFO msg="Upload complete. (238ms)"4494lb # [ 57.017501] niks3-server[1028]: 2026/09/23 04:44:06 INFO Completed upload id=94495lb # [ 57.021157] niks3-server[1028]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4496worker1 # [ 57.052400] nix-grpc-daemon[767]: ts=2026-09-23T04:44:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_in=5124497worker1 # [ 57.067348] nix-grpc-daemon[767]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504498worker1 # [ 57.073941] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4499worker1 # [ 57.079050] nix-daemon[835]: reaped child process 859, status = succeeded4500client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4501worker1 # [ 57.091297] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4502client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4503client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4504client # farm-a> LOG-farm-a-sig4505worker1 # [ 58.327697] nix-daemon[835]: reaped child process 881, status = succeeded4506worker1 # [ 58.437957] nix-daemon[835]: accepted connection from pid 897, user nix-grpc-daemon (trusted)4507worker1 # [ 58.460257] nix-daemon[835]: reaped child process 900, status = succeeded4508lb # [ 58.439550] niks3-server[1028]: 2026/09/23 04:44:08 INFO Received uploads request method=POST path=/api/pending_closures4509worker1 # [ 58.476440] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.050Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4510worker1 # [ 58.477910] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.050Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4511lb # [ 58.479041] niks3-server[1028]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4512lb # [ 58.494411] niks3-server[1028]: 2026/09/23 04:44:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4513worker1 # [ 58.526374] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.101Z level=INFO msg="Uploading 1 narinfos"4514lb # [ 58.509947] niks3-server[1028]: 2026/09/23 04:44:08 INFO Signed narinfos id=10 count=14515lb # [ 58.516784] niks3-server[1028]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4516lb # [ 58.529192] niks3-server[1028]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4517worker1 # [ 58.558617] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.134Z level=INFO msg="Upload complete. (253ms)"4518lb # [ 58.536891] niks3-server[1028]: 2026/09/23 04:44:08 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4519lb # [ 58.542973] niks3-server[1028]: 2026/09/23 04:44:08 INFO Completed upload id=104520lb # [ 58.545822] niks3-server[1028]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4521worker1 # [ 58.569171] nix-grpc-daemon[767]: ts=2026-09-23T04:44:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14522worker2 # [ 58.537386] nix-grpc-daemon[771]: ts=2026-09-23T04:44:08Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04523worker2 # [ 58.543445] nix-grpc-daemon[771]: ts=2026-09-23T04:44:08Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14524worker1 # [ 58.590370] nix-daemon[835]: reaped child process 879, status = succeeded4525worker1 # [ 58.633732] nix-grpc-daemon[767]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504526worker1 # [ 58.650221] nix-grpc-daemon[767]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385504527worker1 # [ 58.664743] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.240Z level=INFO msg="All 1 paths already cached"4528client # copying 1 paths to 'lb:50051'...4529client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4530worker1 # [ 58.675509] nix-grpc-daemon[767]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385504531lb # [ 58.655479] envoy[897]: conn peer=[2001:db8:1::1]:57062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9575 tx=4415 ms=62734532lb # [ 58.660877] envoy[897]: conn peer=[2001:db8:1::1]:56982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=554533worker1 # [ 58.689443] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4534lb # [ 58.665619] envoy[897]: conn peer=[2001:db8:1::1]:56996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1897 ms=19844535worker1 # [ 58.846955] nix-daemon[835]: accepted connection from pid 904, user nix-grpc-daemon (trusted)4536worker1 # [ 58.865420] nix-daemon[835]: reaped child process 907, status = succeeded4537lb # [ 58.845311] niks3-server[1028]: 2026/09/23 04:44:08 INFO Received uploads request method=POST path=/api/pending_closures4538worker1 # [ 58.879299] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.454Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4539worker1 # [ 58.880903] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.454Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4540lb # [ 58.874564] niks3-server[1028]: 2026/09/23 04:44:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4541worker1 # [ 58.906755] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.482Z level=INFO msg="Uploading 1 narinfos"4542lb # [ 58.890442] niks3-server[1028]: 2026/09/23 04:44:08 INFO Signed narinfos id=11 count=14543lb # [ 58.898115] niks3-server[1028]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4544lb # [ 58.912591] niks3-server[1028]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4545worker1 # [ 58.942684] nix-grpc-daemon[861]: time=2026-09-23T04:44:08.518Z level=INFO msg="Upload complete. (236ms)"4546lb # [ 58.918833] niks3-server[1028]: 2026/09/23 04:44:08 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4547lb # [ 58.925161] niks3-server[1028]: 2026/09/23 04:44:08 INFO Completed upload id=114548worker1 # [ 58.952283] nix-grpc-daemon[767]: ts=2026-09-23T04:44:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_in=6244549lb # [ 58.931222] niks3-server[1028]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4550worker1 # [ 58.964483] nix-grpc-daemon[767]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504551worker1 # [ 58.976383] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4552worker1 # [ 58.981569] nix-daemon[835]: reaped child process 902, status = succeeded4553client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4554worker1 # [ 58.992547] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4555client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4556client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4557client # farm-b> LOG-farm-b-sig4558worker1 # [ 59.839369] nix-daemon[835]: reaped child process 911, status = succeeded4559worker1 # [ 59.948801] nix-daemon[835]: accepted connection from pid 927, user nix-grpc-daemon (trusted)4560worker1 # [ 59.972531] nix-daemon[835]: reaped child process 930, status = succeeded4561lb # [ 59.950086] niks3-server[1028]: 2026/09/23 04:44:09 INFO Received uploads request method=POST path=/api/pending_closures4562worker1 # [ 59.985374] nix-grpc-daemon[861]: time=2026-09-23T04:44:09.560Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4563worker1 # [ 59.986888] nix-grpc-daemon[861]: time=2026-09-23T04:44:09.560Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4564worker1 # [ 60.032965] nix-grpc-daemon[861]: time=2026-09-23T04:44:09.608Z level=INFO msg="Uploading 1 narinfos"4565lb # [ 59.996184] niks3-server[1028]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4566lb # [ 60.013167] niks3-server[1028]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4567lb # [ 60.019943] niks3-server[1028]: 2026/09/23 04:44:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4568lb # [ 60.024677] niks3-server[1028]: 2026/09/23 04:44:09 INFO Signed narinfos id=12 count=14569lb # [ 60.031585] niks3-server[1028]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4570worker1 # [ 60.061616] nix-grpc-daemon[861]: time=2026-09-23T04:44:09.637Z level=INFO msg="Upload complete. (257ms)"4571lb # [ 60.039269] niks3-server[1028]: 2026/09/23 04:44:09 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4572lb # [ 60.042647] niks3-server[1028]: 2026/09/23 04:44:09 INFO Completed upload id=124573lb # [ 60.046095] niks3-server[1028]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4574worker1 # [ 60.073306] nix-grpc-daemon[767]: ts=2026-09-23T04:44:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14575worker2 # [ 60.041998] nix-grpc-daemon[771]: ts=2026-09-23T04:44:09Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04576worker2 # [ 60.047995] nix-grpc-daemon[771]: ts=2026-09-23T04:44:09Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14577worker1 # [ 60.093730] nix-daemon[835]: reaped child process 909, status = succeeded4578worker1 # [ 60.129509] nix-grpc-daemon[767]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504579worker1 # [ 60.146295] nix-grpc-daemon[767]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385504580worker1 # [ 60.162026] nix-grpc-daemon[861]: time=2026-09-23T04:44:09.737Z level=INFO msg="All 2 paths already cached"4581client # copying 1 paths to 'lb:50051'...4582worker1 # [ 60.174594] nix-grpc-daemon[767]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385504583client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4584worker1 # [ 60.197656] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4585worker1 # [ 60.363541] nix-daemon[835]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4586worker1 # [ 60.387738] nix-daemon[835]: reaped child process 937, status = succeeded4587lb # [ 60.365060] niks3-server[1028]: 2026/09/23 04:44:09 INFO Received uploads request method=POST path=/api/pending_closures4588worker1 # [ 60.398920] nix-grpc-daemon[861]: time=2026-09-23T04:44:09.974Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4589worker1 # [ 60.400533] nix-grpc-daemon[861]: time=2026-09-23T04:44:09.974Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4590lb # [ 60.395429] niks3-server[1028]: 2026/09/23 04:44:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4591worker1 # [ 60.426583] nix-grpc-daemon[861]: time=2026-09-23T04:44:10.002Z level=INFO msg="Uploading 1 narinfos"4592lb # [ 60.411697] niks3-server[1028]: 2026/09/23 04:44:09 INFO Signed narinfos id=13 count=14593lb # [ 60.420006] niks3-server[1028]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4594lb # [ 60.427614] niks3-server[1028]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4595worker1 # [ 60.461238] nix-grpc-daemon[861]: time=2026-09-23T04:44:10.037Z level=INFO msg="Upload complete. (244ms)"4596lb # [ 60.437355] niks3-server[1028]: 2026/09/23 04:44:09 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4597lb # [ 60.446459] niks3-server[1028]: 2026/09/23 04:44:10 INFO Completed upload id=134598lb # [ 60.449074] niks3-server[1028]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4599worker1 # [ 60.469417] nix-grpc-daemon[767]: ts=2026-09-23T04:44:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_in=7524600worker1 # [ 60.480709] nix-grpc-daemon[767]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504601worker1 # [ 60.491470] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4602worker1 # [ 60.496967] nix-daemon[835]: reaped child process 932, status = succeeded4603client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4604client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4605worker1 # [ 60.510744] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4606client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4607client # farm-top> LOG-farm-top-sig4608worker1 # [ 62.382114] nix-daemon[835]: reaped child process 941, status = succeeded4609worker1 # [ 62.497712] nix-daemon[835]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4610worker1 # [ 62.518021] nix-daemon[835]: reaped child process 960, status = succeeded4611lb # [ 62.498944] niks3-server[1028]: 2026/09/23 04:44:12 INFO Received uploads request method=POST path=/api/pending_closures4612worker1 # [ 62.535922] nix-grpc-daemon[861]: time=2026-09-23T04:44:12.109Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4613worker1 # [ 62.538434] nix-grpc-daemon[861]: time=2026-09-23T04:44:12.109Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4614lb # [ 62.542607] niks3-server[1028]: 2026/09/23 04:44:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4615worker1 # [ 62.577821] nix-grpc-daemon[861]: time=2026-09-23T04:44:12.153Z level=INFO msg="Uploading 1 narinfos"4616lb # [ 62.558587] niks3-server[1028]: 2026/09/23 04:44:12 INFO Signed narinfos id=14 count=14617lb # [ 62.573589] niks3-server[1028]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4618lb # [ 62.584466] niks3-server[1028]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4619lb # [ 62.599590] niks3-server[1028]: 2026/09/23 04:44:12 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4620worker1 # [ 62.631076] nix-grpc-daemon[861]: time=2026-09-23T04:44:12.206Z level=INFO msg="Upload complete. (307ms)"4621lb # [ 62.611411] niks3-server[1028]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4622lb # [ 62.617128] niks3-server[1028]: 2026/09/23 04:44:12 INFO Completed upload id=144623lb # [ 62.619971] niks3-server[1028]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4624worker1 # [ 62.645233] nix-grpc-daemon[767]: ts=2026-09-23T04:44:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=2 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14625worker1 # [ 62.663617] nix-daemon[835]: reaped child process 939, status = succeeded4626client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.40 seconds)4627client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4628client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4629client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4630worker1 # [ 63.010022] nix-grpc-daemon[767]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:602484631worker1 # [ 63.053015] nix-grpc-daemon[767]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:602484632client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds)4633client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4634worker2 # [ 63.273984] nix-grpc-daemon[771]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:546004635client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4636client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4637client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4638client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4639worker2 # [ 63.321661] nix-grpc-daemon[771]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:546004640worker2 # [ 63.331129] nix-grpc-daemon[771]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4641worker2 # [ 63.349489] nix-grpc-daemon[771]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4642worker2 # [ 63.365673] nix-grpc-daemon[771]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4643client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.32 seconds)4644client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4645client # finding garbage collector roots...4646client # removing stale temporary roots file "/nix/var/nix/temproots/848"4647client # removing stale temporary roots file "/nix/var/nix/temproots/939"4648client # removing stale temporary roots file "/nix/var/nix/temproots/916"4649client # removing stale temporary roots file "/nix/var/nix/temproots/903"4650client # removing stale temporary roots file "/nix/var/nix/temproots/883"4651client # removing stale temporary roots file "/nix/var/nix/temproots/924"4652client # removing stale temporary roots file "/nix/var/nix/temproots/895"4653client # removing stale temporary roots file "/nix/var/nix/temproots/862"4654client # deleting specified paths...4655client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4656client # deleting unused links...4657client # note: hard linking is currently saving 0.0 KiB4658worker2 # [ 63.740571] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429564659worker2 # [ 63.748463] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429564660worker2 # [ 63.756366] nix-grpc-daemon[874]: time=2026-09-23T04:44:13.322Z level=INFO msg="All 1 paths already cached"4661worker2 # [ 63.762489] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:429564662worker2 # [ 63.769864] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:429564663worker1 # [ 63.815825] nix-grpc-daemon[767]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:385504664client # copying 3 paths...4665worker1 # [ 63.829815] nix-grpc-daemon[767]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:385504666client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4667worker1 # [ 63.856247] nix-grpc-daemon[767]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:385504668worker1 # [ 63.863734] nix-grpc-daemon[767]: ts=2026-09-23T04:44:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_out=2324669worker2 # [ 63.829232] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:429564670worker2 # [ 63.837076] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_out=1764671worker2 # [ 63.846072] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:429564672worker2 # [ 63.851426] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_out=1284673client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4674client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4675client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.54 seconds)4676(finished: subtest: outputs read back through any worker carry the cache signature, in 7.70 seconds)4677subtest: two clients wanting the same drv build it once4678client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4679client # [ 64.009439] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4680client # Running as unit: dedup1.service; invocation ID: fd40aa34f4494b6ba394f2bd265c8c5d4681client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4682client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4683worker1 # [ 64.238042] nix-grpc-daemon[767]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:385504684client # [ 64.210633] nix[992]: don't know how to build these paths:4685client # [ 64.212600] nix[992]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4686worker2 # [ 64.215182] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04687worker2 # [ 64.222594] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14688worker2 # [ 64.263921] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564689worker2 # [ 64.280848] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429564690client # [ 64.300222] nix[992]: copying 1 paths to 'lb:50051'...4691worker2 # [ 64.295146] nix-grpc-daemon[771]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429564692client # [ 64.305174] nix[992]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4693worker2 # [ 64.310077] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4694worker2 # [ 64.482053] nix-daemon[859]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4695worker2 # [ 64.503606] nix-daemon[859]: reaped child process 1055, status = succeeded4696lb # [ 64.525190] niks3-server[1028]: 2026/09/23 04:44:14 INFO Received uploads request method=POST path=/api/pending_closures4697worker2 # [ 64.517943] nix-grpc-daemon[874]: time=2026-09-23T04:44:14.085Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4698worker2 # [ 64.519643] nix-grpc-daemon[874]: time=2026-09-23T04:44:14.085Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4699lb # [ 64.557269] niks3-server[1028]: 2026/09/23 04:44:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4700worker2 # [ 64.550346] nix-grpc-daemon[874]: time=2026-09-23T04:44:14.118Z level=INFO msg="Uploading 1 narinfos"4701lb # [ 64.571345] niks3-server[1028]: 2026/09/23 04:44:14 INFO Signed narinfos id=15 count=14702lb # [ 64.578866] niks3-server[1028]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4703lb # [ 64.592183] niks3-server[1028]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4704lb # [ 64.602302] niks3-server[1028]: 2026/09/23 04:44:14 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4705worker2 # [ 64.581797] nix-grpc-daemon[874]: time=2026-09-23T04:44:14.149Z level=INFO msg="Upload complete. (254ms)"4706lb # [ 64.608485] niks3-server[1028]: 2026/09/23 04:44:14 INFO Completed upload id=154707worker2 # [ 64.592530] nix-grpc-daemon[771]: ts=2026-09-23T04:44:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_in=4324708lb # [ 64.611122] niks3-server[1028]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4709worker2 # [ 64.607149] nix-grpc-daemon[771]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564710worker2 # [ 64.615072] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4711worker2 # [ 64.620352] nix-daemon[859]: reaped child process 1050, status = succeeded4712client # [ 64.637901] nix[992]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4713worker2 # [ 64.634614] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4714client # [ 65.109532] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4715client # Running as unit: dedup2.service; invocation ID: add7a3f6b4f94f75bbf4e3b9e2017eab4716client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.12 seconds)4717client: must succeed: sleep 24718worker1 # [ 65.404845] nix-grpc-daemon[767]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:385504719client # [ 65.377859] nix[1008]: don't know how to build these paths:4720client # [ 65.381627] nix[1008]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4721worker2 # [ 65.382824] nix-grpc-daemon[771]: ts=2026-09-23T04:44:14Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04722worker2 # [ 65.385350] nix-grpc-daemon[771]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564723client # [ 65.396231] nix[1008]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4724client: (finished: must succeed: sleep 2, in 2.06 seconds)4725client: waiting for success: ! systemctl is-active dedup1 dedup24726worker2 # [ 67.684534] nix-daemon[859]: reaped child process 1059, status = succeeded4727worker2 # [ 67.802570] nix-daemon[859]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)4728worker2 # [ 67.825269] nix-daemon[859]: reaped child process 1090, status = succeeded4729lb # [ 67.847741] niks3-server[1028]: 2026/09/23 04:44:17 INFO Received uploads request method=POST path=/api/pending_closures4730worker2 # [ 67.876712] nix-grpc-daemon[874]: time=2026-09-23T04:44:17.440Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4731worker2 # [ 67.879551] nix-grpc-daemon[874]: time=2026-09-23T04:44:17.440Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4732worker2 # [ 67.922507] nix-grpc-daemon[874]: time=2026-09-23T04:44:17.490Z level=INFO msg="Uploading 1 narinfos"4733lb # [ 67.929610] niks3-server[1028]: 2026/09/23 04:44:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4734lb # [ 67.944406] niks3-server[1028]: 2026/09/23 04:44:17 INFO Signed narinfos id=16 count=14735lb # [ 67.953818] niks3-server[1028]: 2026/09/23 04:44:17 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4736lb # [ 67.963865] niks3-server[1028]: 2026/09/23 04:44:17 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4737lb # [ 67.977343] niks3-server[1028]: 2026/09/23 04:44:17 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4738lb # [ 67.983742] niks3-server[1028]: 2026/09/23 04:44:17 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4739lb # [ 67.996424] niks3-server[1028]: 2026/09/23 04:44:17 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4740worker2 # [ 67.985065] nix-grpc-daemon[874]: time=2026-09-23T04:44:17.553Z level=INFO msg="Upload complete. (339ms)"4741lb # [ 68.005754] niks3-server[1028]: 2026/09/23 04:44:17 INFO Completed upload id=164742worker2 # [ 68.001200] nix-grpc-daemon[771]: ts=2026-09-23T04:44:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14743worker2 # [ 68.020293] nix-grpc-daemon[771]: ts=2026-09-23T04:44:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=2 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14744worker2 # [ 68.029075] nix-daemon[859]: reaped child process 1057, status = succeeded4745client # [ 68.066249] systemd[1]: dedup1.service: Deactivated successfully.4746client # [ 68.069481] systemd[1]: dedup1.service: Consumed 174ms CPU time over 4.059s wall clock time, 39.5M memory peak, 7.3K incoming IP traffic, 8.6K outgoing IP traffic.4747client # [ 68.076796] systemd[1]: dedup2.service: Deactivated successfully.4748client # [ 68.079856] systemd[1]: dedup2.service: Consumed 224ms CPU time over 2.971s wall clock time, 35.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4749client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4750client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4751lb # [ 68.656477] envoy[897]: conn peer=[2001:db8:1::1]:56998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9571 tx=4409 ms=61584752lb # [ 68.656717] envoy[897]: conn peer=[2001:db8:1::1]:33130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2932 ms=1844753lb # [ 68.656818] envoy[897]: conn peer=[2001:db8:1::1]:33150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=824754lb # [ 68.656914] envoy[897]: conn peer=[2001:db8:1::1]:33134 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=1104755lb # [ 68.657008] envoy[897]: conn peer=[2001:db8:1::1]:33148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=1014756lb # [ 68.657197] envoy[897]: conn peer=[2001:db8:1::1]:33154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3035 tx=1557 ms=38154757lb # [ 68.657295] envoy[897]: conn peer=[2001:db8:1::1]:33168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=26744758client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4759worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4760worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4761worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4762worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4763worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4764worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4765worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4766worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4767(finished: subtest: two clients wanting the same drv build it once, in 5.00 seconds)4768subtest: --timeout and --max-silent-time reach the builder4769worker2 # [ 69.166398] nix-grpc-daemon[771]: ts=2026-09-23T04:44:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429564770worker2 # [ 69.184495] nix-grpc-daemon[771]: ts=2026-09-23T04:44:18Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04771worker1 # [ 69.232740] nix-grpc-daemon[767]: ts=2026-09-23T04:44:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504772worker2 # [ 69.190607] nix-grpc-daemon[771]: ts=2026-09-23T04:44:18Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14773worker1 # [ 69.248788] nix-grpc-daemon[767]: ts=2026-09-23T04:44:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385504774worker1 # [ 69.263181] nix-grpc-daemon[767]: ts=2026-09-23T04:44:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385504775worker1 # [ 69.278193] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4776worker1 # [ 69.438818] nix-daemon[835]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4777worker1 # [ 69.461337] nix-daemon[835]: reaped child process 984, status = succeeded4778lb # [ 69.441138] niks3-server[1028]: 2026/09/23 04:44:19 INFO Received uploads request method=POST path=/api/pending_closures4779worker1 # [ 69.476500] nix-grpc-daemon[861]: time=2026-09-23T04:44:19.051Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4780worker1 # [ 69.478569] nix-grpc-daemon[861]: time=2026-09-23T04:44:19.051Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4781lb # [ 69.481637] niks3-server[1028]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4782worker1 # [ 69.514740] nix-grpc-daemon[861]: time=2026-09-23T04:44:19.090Z level=INFO msg="Uploading 1 narinfos"4783lb # [ 69.493558] niks3-server[1028]: 2026/09/23 04:44:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4784lb # [ 69.501484] niks3-server[1028]: 2026/09/23 04:44:19 INFO Signed narinfos id=17 count=14785lb # [ 69.509753] niks3-server[1028]: 2026/09/23 04:44:19 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4786lb # [ 69.514614] niks3-server[1028]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4787worker1 # [ 69.545567] nix-grpc-daemon[861]: time=2026-09-23T04:44:19.121Z level=INFO msg="Upload complete. (250ms)"4788lb # [ 69.523464] niks3-server[1028]: 2026/09/23 04:44:19 INFO Completed upload id=174789lb # [ 69.526471] niks3-server[1028]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4790worker1 # [ 69.556332] nix-grpc-daemon[767]: ts=2026-09-23T04:44:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_in=4244791worker1 # [ 69.576546] nix-grpc-daemon[767]: ts=2026-09-23T04:44:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504792worker1 # [ 69.584238] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4793worker1 # [ 69.590087] nix-daemon[835]: reaped child process 979, status = succeeded4794worker1 # [ 69.604120] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4795worker1 # [ 75.618641] nix-grpc-daemon[767]: ts=2026-09-23T04:44:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04796worker1 # [ 75.635519] nix-daemon[835]: reaped child process 986, status = succeeded4797worker1 # [ 75.636716] nix-daemon[835]: reaped child process 988, status = succeeded4798worker2 # [ 75.811736] nix-grpc-daemon[771]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429564799worker2 # [ 75.826283] nix-grpc-daemon[771]: ts=2026-09-23T04:44:25Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04800worker2 # [ 75.834183] nix-grpc-daemon[771]: ts=2026-09-23T04:44:25Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14801worker2 # [ 75.843814] nix-grpc-daemon[771]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564802worker2 # [ 75.855298] nix-grpc-daemon[771]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429564803worker2 # [ 75.871998] nix-grpc-daemon[771]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429564804worker2 # [ 75.887606] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4805worker2 # [ 76.049107] nix-daemon[859]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4806worker2 # [ 76.068097] nix-daemon[859]: reaped child process 1110, status = succeeded4807lb # [ 76.091889] niks3-server[1028]: 2026/09/23 04:44:25 INFO Received uploads request method=POST path=/api/pending_closures4808worker2 # [ 76.084566] nix-grpc-daemon[874]: time=2026-09-23T04:44:25.652Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4809worker2 # [ 76.086601] nix-grpc-daemon[874]: time=2026-09-23T04:44:25.652Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4810lb # [ 76.124526] niks3-server[1028]: 2026/09/23 04:44:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4811worker2 # [ 76.116865] nix-grpc-daemon[874]: time=2026-09-23T04:44:25.684Z level=INFO msg="Uploading 1 narinfos"4812lb # [ 76.136606] niks3-server[1028]: 2026/09/23 04:44:25 INFO Signed narinfos id=18 count=14813lb # [ 76.146792] niks3-server[1028]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4814lb # [ 76.164503] niks3-server[1028]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4815worker2 # [ 76.157495] nix-grpc-daemon[874]: time=2026-09-23T04:44:25.725Z level=INFO msg="Upload complete. (253ms)"4816lb # [ 76.176803] niks3-server[1028]: 2026/09/23 04:44:25 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4817lb # [ 76.181971] niks3-server[1028]: 2026/09/23 04:44:25 INFO Completed upload id=184818lb # [ 76.185566] niks3-server[1028]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4819worker2 # [ 76.169330] nix-grpc-daemon[771]: ts=2026-09-23T04:44:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_in=4244820worker2 # [ 76.184675] nix-grpc-daemon[771]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564821worker2 # [ 76.193632] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4822worker2 # [ 76.203517] nix-daemon[859]: reaped child process 1105, status = succeeded4823worker2 # [ 76.210845] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4824lb # [ 78.654515] envoy[897]: conn peer=[2001:db8:1::1]:33178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2971 tx=934 ms=64304825worker2 # [ 82.233589] nix-grpc-daemon[771]: ts=2026-09-23T04:44:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04826worker2 # [ 82.284316] nix-daemon[859]: reaped child process 1112, status = succeeded4827worker2 # [ 82.286618] nix-daemon[859]: reaped child process 1114, status = succeeded4828(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.56 seconds)4829subtest: the first client leaving does not fail the second4830client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4831client # [ 82.587940] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4832client # Running as unit: share1.service; invocation ID: 8414c7c88d9f4f56b54fa10d7ce50d914833client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.11 seconds)4834client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4835client # [ 82.840895] nix[1081]: don't know how to build these paths:4836client # [ 82.841861] nix[1081]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4837worker2 # [ 82.831039] nix-grpc-daemon[771]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429564838worker2 # [ 82.850531] nix-grpc-daemon[771]: ts=2026-09-23T04:44:32Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04839worker2 # [ 82.857469] nix-grpc-daemon[771]: ts=2026-09-23T04:44:32Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14840worker1 # [ 82.907504] nix-grpc-daemon[767]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504841worker1 # [ 82.927950] nix-grpc-daemon[767]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385504842client # [ 82.907698] nix[1081]: copying 1 paths to 'lb:50051'...4843client # [ 82.912389] nix[1081]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4844worker1 # [ 82.947855] nix-grpc-daemon[767]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385504845worker1 # [ 82.966593] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4846worker1 # [ 83.157855] nix-daemon[835]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4847worker1 # [ 83.181117] nix-daemon[835]: reaped child process 1014, status = succeeded4848lb # [ 83.162124] niks3-server[1028]: 2026/09/23 04:44:32 INFO Received uploads request method=POST path=/api/pending_closures4849worker1 # [ 83.201559] nix-grpc-daemon[861]: time=2026-09-23T04:44:32.776Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4850worker1 # [ 83.203201] nix-grpc-daemon[861]: time=2026-09-23T04:44:32.776Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4851lb # [ 83.206040] niks3-server[1028]: 2026/09/23 04:44:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4852worker1 # [ 83.243187] nix-grpc-daemon[861]: time=2026-09-23T04:44:32.818Z level=INFO msg="Uploading 1 narinfos"4853lb # [ 83.222176] niks3-server[1028]: 2026/09/23 04:44:32 INFO Signed narinfos id=19 count=14854lb # [ 83.239167] niks3-server[1028]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4855lb # [ 83.249850] niks3-server[1028]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4856lb # [ 83.267484] niks3-server[1028]: 2026/09/23 04:44:32 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4857lb # [ 83.276323] niks3-server[1028]: 2026/09/23 04:44:32 INFO Completed upload id=194858worker1 # [ 83.299320] nix-grpc-daemon[861]: time=2026-09-23T04:44:32.875Z level=INFO msg="Upload complete. (312ms)"4859lb # [ 83.282619] niks3-server[1028]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4860worker1 # [ 83.313481] nix-grpc-daemon[767]: ts=2026-09-23T04:44:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_in=4324861worker1 # [ 83.324895] nix-grpc-daemon[767]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504862worker1 # [ 83.342666] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4863worker1 # [ 83.349077] nix-daemon[835]: reaped child process 1009, status = succeeded4864worker1 # [ 83.364872] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4865client # [ 83.329024] nix[1081]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4866client # [ 83.719253] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4867client # Running as unit: share2.service; invocation ID: e9bcc32e778b4887ba106a863ed72e424868client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.14 seconds)4869client # [ 83.998712] nix[1097]: don't know how to build these paths:4870client # [ 83.999836] nix[1097]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4871worker2 # [ 83.989380] nix-grpc-daemon[771]: ts=2026-09-23T04:44:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429564872worker2 # [ 84.005665] nix-grpc-daemon[771]: ts=2026-09-23T04:44:33Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04873client # [ 84.021274] nix[1097]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4874worker1 # [ 84.055691] nix-grpc-daemon[767]: ts=2026-09-23T04:44:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385504875client: must succeed: sleep 2; systemctl kill -s INT share14876client # [ 87.143358] systemd[1]: share1.service: Sent signal SIGINT to main process 1081 (nix) on client request.4877client # [ 87.150544] nix[1081]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4878client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4879client: waiting for success: ! systemctl is-active share14880client # [ 87.164889] nix[1081]: error: interrupted by the user4881client # [ 87.228428] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4882client # [ 87.231559] systemd[1]: share1.service: Failed with result 'exit-code'.4883client # [ 87.234351] systemd[1]: share1.service: Consumed 211ms CPU time over 4.643s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8.3K outgoing IP traffic.4884client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4885client: waiting for success: ! systemctl is-active share24886worker1 # [ 88.606051] nix-daemon[835]: reaped child process 1018, status = succeeded4887lb # [ 88.656707] envoy[897]: conn peer=[2001:db8:1::1]:52250 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3002 tx=954 ms=64844888lb # [ 88.661765] envoy[897]: conn peer=[2001:db8:1::1]:41784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3003 tx=788 ms=43354889worker1 # [ 88.712130] nix-daemon[835]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4890worker1 # [ 88.732917] nix-daemon[835]: reaped child process 1055, status = succeeded4891lb # [ 88.707526] niks3-server[1028]: 2026/09/23 04:44:38 INFO Received uploads request method=POST path=/api/pending_closures4892worker1 # [ 88.744489] nix-grpc-daemon[861]: time=2026-09-23T04:44:38.318Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4893worker1 # [ 88.748134] nix-grpc-daemon[861]: time=2026-09-23T04:44:38.318Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4894lb # [ 88.746404] niks3-server[1028]: 2026/09/23 04:44:38 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4895worker1 # [ 88.794372] nix-grpc-daemon[861]: time=2026-09-23T04:44:38.369Z level=INFO msg="Uploading 1 narinfos"4896lb # [ 88.764700] niks3-server[1028]: 2026/09/23 04:44:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4897lb # [ 88.775277] niks3-server[1028]: 2026/09/23 04:44:38 INFO Signed narinfos id=20 count=14898lb # [ 88.780836] niks3-server[1028]: 2026/09/23 04:44:38 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4899lb # [ 88.787919] niks3-server[1028]: 2026/09/23 04:44:38 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4900lb # [ 88.796821] niks3-server[1028]: 2026/09/23 04:44:38 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4901lb # [ 88.803072] niks3-server[1028]: 2026/09/23 04:44:38 INFO Completed upload id=204902lb # [ 88.807205] niks3-server[1028]: 2026/09/23 04:44:38 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4903worker1 # [ 88.826597] nix-grpc-daemon[861]: time=2026-09-23T04:44:38.402Z level=INFO msg="Upload complete. (242ms)"4904worker1 # [ 88.837295] nix-grpc-daemon[767]: ts=2026-09-23T04:44:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14905worker1 # [ 88.855381] nix-grpc-daemon[767]: ts=2026-09-23T04:44:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14906worker1 # [ 88.861432] nix-daemon[835]: reaped child process 1016, status = succeeded4907client # [ 88.881441] systemd[1]: share2.service: Deactivated successfully.4908client # [ 88.886523] systemd[1]: share2.service: Consumed 247ms CPU time over 5.164s wall clock time, 33.3M memory peak, 6K incoming IP traffic, 5.8K outgoing IP traffic.4909client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4910client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4911client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4912(finished: subtest: the first client leaving does not fail the second, in 7.19 seconds)4913subtest: build hook: nix-daemon with builders = grpc://lb4914client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4915client # [ 89.836683] systemd[1]: Started Nix Daemon.4916client # [ 89.953441] nix-daemon[1149]: accepted connection from pid 1146, user root (trusted)4917worker2 # [ 90.095290] nix-grpc-daemon[771]: ts=2026-09-23T04:44:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:429564918worker1 # [ 90.141148] nix-grpc-daemon[767]: ts=2026-09-23T04:44:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385504919worker2 # [ 90.119519] nix-grpc-daemon[771]: ts=2026-09-23T04:44:39Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04920worker2 # [ 90.127285] nix-grpc-daemon[771]: ts=2026-09-23T04:44:39Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14921worker2 # [ 90.136621] nix-grpc-daemon[771]: ts=2026-09-23T04:44:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564922worker2 # [ 90.147928] nix-grpc-daemon[771]: ts=2026-09-23T04:44:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429564923worker2 # [ 90.162335] nix-grpc-daemon[771]: ts=2026-09-23T04:44:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429564924worker2 # [ 90.178057] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4925worker2 # [ 90.337303] nix-daemon[859]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)4926worker2 # [ 90.356882] nix-daemon[859]: reaped child process 1162, status = succeeded4927lb # [ 90.379213] niks3-server[1028]: 2026/09/23 04:44:39 INFO Received uploads request method=POST path=/api/pending_closures4928worker2 # [ 90.372363] nix-grpc-daemon[874]: time=2026-09-23T04:44:39.939Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4929worker2 # [ 90.373831] nix-grpc-daemon[874]: time=2026-09-23T04:44:39.940Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4930worker2 # [ 90.405178] nix-grpc-daemon[874]: time=2026-09-23T04:44:39.972Z level=INFO msg="Uploading 1 narinfos"4931lb # [ 90.411970] niks3-server[1028]: 2026/09/23 04:44:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4932lb # [ 90.426613] niks3-server[1028]: 2026/09/23 04:44:39 INFO Signed narinfos id=21 count=14933lb # [ 90.437077] niks3-server[1028]: 2026/09/23 04:44:39 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4934lb # [ 90.452184] niks3-server[1028]: 2026/09/23 04:44:40 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4935worker2 # [ 90.444786] nix-grpc-daemon[874]: time=2026-09-23T04:44:40.012Z level=INFO msg="Upload complete. (248ms)"4936lb # [ 90.465743] niks3-server[1028]: 2026/09/23 04:44:40 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4937worker2 # [ 90.452142] nix-grpc-daemon[771]: ts=2026-09-23T04:44:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_in=5124938lb # [ 90.472145] niks3-server[1028]: 2026/09/23 04:44:40 INFO Completed upload id=214939lb # [ 90.476682] niks3-server[1028]: 2026/09/23 04:44:40 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4940worker2 # [ 90.467455] nix-grpc-daemon[771]: ts=2026-09-23T04:44:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564941worker2 # [ 90.475413] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4942worker2 # [ 90.481046] nix-daemon[859]: reaped child process 1157, status = succeeded4943worker2 # [ 90.493946] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4944worker2 # [ 91.398199] nix-daemon[859]: reaped child process 1166, status = succeeded4945worker2 # [ 91.467934] nix-daemon[859]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)4946worker2 # [ 91.486221] nix-daemon[859]: reaped child process 1185, status = succeeded4947lb # [ 91.509608] niks3-server[1028]: 2026/09/23 04:44:41 INFO Received uploads request method=POST path=/api/pending_closures4948worker2 # [ 91.503517] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.068Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4949worker2 # [ 91.505432] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.068Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4950worker2 # [ 91.539582] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.107Z level=INFO msg="Uploading 1 narinfos"4951lb # [ 91.547331] niks3-server[1028]: 2026/09/23 04:44:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4952lb # [ 91.560885] niks3-server[1028]: 2026/09/23 04:44:41 INFO Signed narinfos id=22 count=14953lb # [ 91.569342] niks3-server[1028]: 2026/09/23 04:44:41 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4954lb # [ 91.580843] niks3-server[1028]: 2026/09/23 04:44:41 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4955lb # [ 91.593294] niks3-server[1028]: 2026/09/23 04:44:41 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4956worker2 # [ 91.580441] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.148Z level=INFO msg="Upload complete. (276ms)"4957lb # [ 91.598431] niks3-server[1028]: 2026/09/23 04:44:41 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4958lb # [ 91.606951] niks3-server[1028]: 2026/09/23 04:44:41 INFO Completed upload id=224959lb # [ 91.610041] niks3-server[1028]: 2026/09/23 04:44:41 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4960worker2 # [ 91.593166] nix-grpc-daemon[771]: ts=2026-09-23T04:44:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14961worker1 # [ 91.641516] nix-grpc-daemon[767]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:385504962worker2 # [ 91.613155] nix-daemon[859]: reaped child process 1164, status = succeeded4963worker1 # [ 91.654919] nix-grpc-daemon[767]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4964worker1 # [ 91.699208] nix-grpc-daemon[767]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:385504965worker1 # [ 91.706201] nix-grpc-daemon[767]: ts=2026-09-23T04:44:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_out=1284966worker1 # [ 91.985878] nix-grpc-daemon[767]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:385504967worker1 # [ 91.993490] nix-grpc-daemon[767]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385504968worker1 # [ 92.001002] nix-grpc-daemon[861]: time=2026-09-23T04:44:41.575Z level=INFO msg="All 1 paths already cached"4969worker2 # [ 91.970834] nix-grpc-daemon[771]: ts=2026-09-23T04:44:41Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04970worker2 # [ 91.978353] nix-grpc-daemon[771]: ts=2026-09-23T04:44:41Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14971worker2 # [ 91.987754] nix-grpc-daemon[771]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564972worker2 # [ 91.999375] nix-grpc-daemon[771]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429564973worker2 # [ 92.014361] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.582Z level=INFO msg="All 1 paths already cached"4974worker2 # [ 92.024112] nix-grpc-daemon[771]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429564975worker2 # [ 92.038398] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4976worker2 # [ 92.191317] nix-daemon[859]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)4977worker2 # [ 92.214973] nix-daemon[859]: reaped child process 1194, status = succeeded4978lb # [ 92.229135] niks3-server[1028]: 2026/09/23 04:44:41 INFO Received uploads request method=POST path=/api/pending_closures4979worker2 # [ 92.222532] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.790Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4980worker2 # [ 92.224578] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.790Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4981lb # [ 92.260590] niks3-server[1028]: 2026/09/23 04:44:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4982worker2 # [ 92.251044] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.818Z level=INFO msg="Uploading 1 narinfos"4983lb # [ 92.274652] niks3-server[1028]: 2026/09/23 04:44:41 INFO Signed narinfos id=23 count=14984lb # [ 92.282320] niks3-server[1028]: 2026/09/23 04:44:41 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4985lb # [ 92.294318] niks3-server[1028]: 2026/09/23 04:44:41 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4986lb # [ 92.300202] niks3-server[1028]: 2026/09/23 04:44:41 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4987lb # [ 92.304146] niks3-server[1028]: 2026/09/23 04:44:41 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4988lb # [ 92.309307] niks3-server[1028]: 2026/09/23 04:44:41 INFO Completed upload id=234989worker2 # [ 92.291343] nix-grpc-daemon[874]: time=2026-09-23T04:44:41.859Z level=INFO msg="Upload complete. (235ms)"4990worker2 # [ 92.300310] nix-grpc-daemon[771]: ts=2026-09-23T04:44:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_in=6244991worker2 # [ 92.312799] nix-grpc-daemon[771]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429564992worker2 # [ 92.325677] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4993worker2 # [ 92.331436] nix-daemon[859]: reaped child process 1189, status = succeeded4994worker2 # [ 92.347323] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4995worker2 # [ 93.458809] nix-daemon[859]: reaped child process 1198, status = succeeded4996worker2 # [ 93.532254] nix-daemon[859]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)4997worker2 # [ 93.549035] nix-daemon[859]: reaped child process 1217, status = succeeded4998lb # [ 93.573777] niks3-server[1028]: 2026/09/23 04:44:43 INFO Received uploads request method=POST path=/api/pending_closures4999worker2 # [ 93.568383] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.135Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5000worker2 # [ 93.570106] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.135Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"5001lb # [ 93.616241] niks3-server[1028]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv5002worker2 # [ 93.612501] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.180Z level=INFO msg="Uploading 1 narinfos"5003lb # [ 93.629255] niks3-server[1028]: 2026/09/23 04:44:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5004lb # [ 93.642370] niks3-server[1028]: 2026/09/23 04:44:43 INFO Signed narinfos id=24 count=15005lb # [ 93.648301] niks3-server[1028]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst5006worker2 # [ 93.644625] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.212Z level=INFO msg="Upload complete. (284ms)"5007lb # [ 93.661915] niks3-server[1028]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls5008lb # [ 93.668983] niks3-server[1028]: 2026/09/23 04:44:43 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5009lb # [ 93.673870] niks3-server[1028]: 2026/09/23 04:44:43 INFO Completed upload id=245010lb # [ 93.677526] niks3-server[1028]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo5011worker2 # [ 93.658179] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=15012worker2 # [ 93.680480] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:429565013worker2 # [ 93.685406] nix-daemon[859]: reaped child process 1196, status = succeeded5014worker1 # [ 93.739313] nix-grpc-daemon[767]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:385505015worker1 # [ 93.746897] nix-grpc-daemon[767]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...5016worker1 # [ 93.762090] nix-grpc-daemon[767]: ts=2026-09-23T04:44:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_out=1765017worker2 # [ 93.997285] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:429565018worker2 # [ 94.005522] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429565019worker2 # [ 94.013561] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.579Z level=INFO msg="All 2 paths already cached"5020worker2 # [ 94.025250] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=05021worker2 # [ 94.034303] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=15022worker2 # [ 94.042824] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429565023worker2 # [ 94.050487] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429565024worker2 # [ 94.061111] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.629Z level=INFO msg="All 2 paths already cached"5025worker2 # [ 94.070786] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429565026worker2 # [ 94.084867] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5027worker2 # [ 94.238460] nix-daemon[859]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5028worker2 # [ 94.258148] nix-daemon[859]: reaped child process 1226, status = succeeded5029lb # [ 94.277889] niks3-server[1028]: 2026/09/23 04:44:43 INFO Received uploads request method=POST path=/api/pending_closures5030worker2 # [ 94.270814] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.838Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5031worker2 # [ 94.272437] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.838Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"5032lb # [ 94.308055] niks3-server[1028]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls5033worker2 # [ 94.299656] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.867Z level=INFO msg="Uploading 1 narinfos"5034lb # [ 94.319830] niks3-server[1028]: 2026/09/23 04:44:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5035lb # [ 94.332505] niks3-server[1028]: 2026/09/23 04:44:43 INFO Signed narinfos id=25 count=15036lb # [ 94.339473] niks3-server[1028]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst5037worker2 # [ 94.334890] nix-grpc-daemon[874]: time=2026-09-23T04:44:43.902Z level=INFO msg="Upload complete. (235ms)"5038lb # [ 94.352404] niks3-server[1028]: 2026/09/23 04:44:43 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5039lb # [ 94.361192] niks3-server[1028]: 2026/09/23 04:44:43 INFO Completed upload id=255040lb # [ 94.366853] niks3-server[1028]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo5041worker2 # [ 94.343041] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_in=7525042worker2 # [ 94.359726] nix-grpc-daemon[771]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429565043worker2 # [ 94.366739] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5044worker2 # [ 94.373353] nix-daemon[859]: reaped child process 1221, status = succeeded5045worker2 # [ 94.388406] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5046worker2 # [ 95.280075] nix-daemon[859]: reaped child process 1230, status = succeeded5047worker2 # [ 95.358276] nix-daemon[859]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5048worker2 # [ 95.377990] nix-daemon[859]: reaped child process 1249, status = succeeded5049lb # [ 95.394370] niks3-server[1028]: 2026/09/23 04:44:44 INFO Received uploads request method=POST path=/api/pending_closures5050worker2 # [ 95.388408] nix-grpc-daemon[874]: time=2026-09-23T04:44:44.955Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5051worker2 # [ 95.389889] nix-grpc-daemon[874]: time=2026-09-23T04:44:44.956Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"5052lb # [ 95.444569] niks3-server[1028]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls5053worker2 # [ 95.435934] nix-grpc-daemon[874]: time=2026-09-23T04:44:45.003Z level=INFO msg="Uploading 1 narinfos"5054lb # [ 95.458117] niks3-server[1028]: 2026/09/23 04:44:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5055lb # [ 95.467531] niks3-server[1028]: 2026/09/23 04:44:45 INFO Signed narinfos id=26 count=15056lb # [ 95.472333] niks3-server[1028]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst5057lb # [ 95.475817] niks3-server[1028]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv5058lb # [ 95.486084] niks3-server[1028]: 2026/09/23 04:44:45 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5059lb # [ 95.490650] niks3-server[1028]: 2026/09/23 04:44:45 INFO Completed upload id=265060worker2 # [ 95.473653] nix-grpc-daemon[874]: time=2026-09-23T04:44:45.041Z level=INFO msg="Upload complete. (274ms)"5061lb # [ 95.494007] niks3-server[1028]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo5062worker2 # [ 95.489749] nix-grpc-daemon[771]: ts=2026-09-23T04:44:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=15063worker2 # [ 95.504375] nix-grpc-daemon[771]: ts=2026-09-23T04:44:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:429565064worker2 # [ 95.510657] nix-daemon[859]: reaped child process 1228, status = succeeded5065worker1 # [ 95.565483] nix-grpc-daemon[767]: ts=2026-09-23T04:44:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:385505066worker1 # [ 95.578088] nix-grpc-daemon[767]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...5067worker1 # [ 95.594260] nix-grpc-daemon[767]: ts=2026-09-23T04:44:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_out=2325068client # [ 95.688473] nix-daemon[1149]: reaped child process 1151, status = succeeded5069client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.01 seconds)5070client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top5071client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)5072client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5073client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)5074client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5075client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5076(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.13 seconds)5077subtest: an input only one worker has still reaches the builder5078client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only5079client # this derivation will be built:5080client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv5081client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...5082client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)5083client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure5084client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)5085worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5086worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)5087worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only5088worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)5089client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&25090client # [ 96.635627] nix-daemon[1149]: accepted connection from pid 1232, user root (trusted)5091client # this derivation will be built:5092client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5093client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5094worker1 # [ 96.823235] nix-grpc-daemon[767]: ts=2026-09-23T04:44:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:385505095worker2 # [ 96.786475] nix-grpc-daemon[771]: ts=2026-09-23T04:44:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429565096client # copying 1 paths...5097client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...5098worker2 # [ 96.802683] nix-grpc-daemon[771]: ts=2026-09-23T04:44:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429565099worker2 # [ 96.818171] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5100worker2 # [ 96.988715] nix-daemon[859]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5101worker2 # [ 97.011078] nix-daemon[859]: reaped child process 1259, status = succeeded5102lb # [ 97.027847] niks3-server[1028]: 2026/09/23 04:44:46 INFO Received uploads request method=POST path=/api/pending_closures5103worker2 # [ 97.020475] nix-grpc-daemon[874]: time=2026-09-23T04:44:46.588Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5104worker2 # [ 97.021944] nix-grpc-daemon[874]: time=2026-09-23T04:44:46.588Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"5105lb # [ 97.060412] niks3-server[1028]: 2026/09/23 04:44:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5106worker2 # [ 97.056900] nix-grpc-daemon[874]: time=2026-09-23T04:44:46.624Z level=INFO msg="Uploading 1 narinfos"5107lb # [ 97.077498] niks3-server[1028]: 2026/09/23 04:44:46 INFO Signed narinfos id=27 count=15108lb # [ 97.086010] niks3-server[1028]: 2026/09/23 04:44:46 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5109lb # [ 97.099528] niks3-server[1028]: 2026/09/23 04:44:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls5110worker2 # [ 97.090537] nix-grpc-daemon[874]: time=2026-09-23T04:44:46.658Z level=INFO msg="Upload complete. (255ms)"5111lb # [ 97.107869] niks3-server[1028]: 2026/09/23 04:44:46 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5112lb # [ 97.113861] niks3-server[1028]: 2026/09/23 04:44:46 INFO Completed upload id=275113worker2 # [ 97.100339] nix-grpc-daemon[771]: ts=2026-09-23T04:44:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_in=1205114lb # [ 97.118561] niks3-server[1028]: 2026/09/23 04:44:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo5115worker2 # [ 97.117790] nix-grpc-daemon[771]: ts=2026-09-23T04:44:46Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=05116worker2 # [ 97.126358] nix-grpc-daemon[771]: ts=2026-09-23T04:44:46Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15117worker2 # [ 97.134099] nix-daemon[859]: reaped child process 1254, status = succeeded5118worker1 # [ 97.205738] nix-grpc-daemon[767]: ts=2026-09-23T04:44:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385505119worker1 # [ 97.221380] nix-grpc-daemon[767]: ts=2026-09-23T04:44:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385505120client # copying 2 paths to 'lb:50051'...5121worker1 # [ 97.240482] nix-grpc-daemon[767]: ts=2026-09-23T04:44:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385505122client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...5123client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...5124worker1 # [ 97.255960] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5125worker1 # [ 97.415451] nix-daemon[835]: accepted connection from pid 1067, user nix-grpc-daemon (trusted)5126worker1 # [ 97.437749] nix-daemon[835]: reaped child process 1070, status = succeeded5127lb # [ 97.410483] niks3-server[1028]: 2026/09/23 04:44:46 INFO Received uploads request method=POST path=/api/pending_closures5128lb # [ 97.420833] niks3-server[1028]: 2026/09/23 04:44:46 INFO Received uploads request method=POST path=/api/pending_closures5129worker1 # [ 97.456293] nix-grpc-daemon[861]: time=2026-09-23T04:44:47.029Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5130worker1 # [ 97.458850] nix-grpc-daemon[861]: time=2026-09-23T04:44:47.029Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"5131worker1 # [ 97.461411] nix-grpc-daemon[861]: time=2026-09-23T04:44:47.029Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"5132lb # [ 97.467004] niks3-server[1028]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls5133worker1 # [ 97.504190] nix-grpc-daemon[861]: time=2026-09-23T04:44:47.079Z level=INFO msg="Uploading 3 narinfos"5134lb # [ 97.480605] niks3-server[1028]: 2026/09/23 04:44:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5135lb # [ 97.491913] niks3-server[1028]: 2026/09/23 04:44:47 INFO Signed narinfos id=28 count=15136lb # [ 97.498884] niks3-server[1028]: 2026/09/23 04:44:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5137lb # [ 97.507762] niks3-server[1028]: 2026/09/23 04:44:47 INFO Signed narinfos id=29 count=25138lb # [ 97.512957] niks3-server[1028]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5139lb # [ 97.515941] niks3-server[1028]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst5140lb # [ 97.529937] niks3-server[1028]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst5141lb # [ 97.539858] niks3-server[1028]: 2026/09/23 04:44:47 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5142lb # [ 97.543968] niks3-server[1028]: 2026/09/23 04:44:47 INFO Completed upload id=295143worker1 # [ 97.568108] nix-grpc-daemon[861]: time=2026-09-23T04:44:47.143Z level=INFO msg="Upload complete. (293ms)"5144lb # [ 97.546853] niks3-server[1028]: 2026/09/23 04:44:47 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5145lb # [ 97.549402] niks3-server[1028]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5146lb # [ 97.552413] niks3-server[1028]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5147lb # [ 97.555543] niks3-server[1028]: 2026/09/23 04:44:47 INFO Completed upload id=285148lb # [ 97.557369] niks3-server[1028]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5149worker1 # [ 97.581262] nix-grpc-daemon[767]: ts=2026-09-23T04:44:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=2 nar_bytes_in=9845150worker1 # [ 97.596404] nix-grpc-daemon[767]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385505151worker1 # [ 97.607003] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5152worker1 # [ 97.613290] nix-daemon[835]: reaped child process 1065, status = succeeded5153client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5154worker1 # [ 97.626883] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5155client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5156client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5157lb # [ 98.656731] envoy[897]: conn peer=[2001:db8:1::1]:41798 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1456 tx=1194 ms=48675158lb # [ 98.661733] envoy[897]: conn peer=[2001:db8:1::1]:41814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1943 ms=16305159lb # [ 98.666569] envoy[897]: conn peer=[2001:db8:1::1]:45824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=765160lb # [ 98.671264] envoy[897]: conn peer=[2001:db8:1::1]:45840 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3912 tx=2156 ms=18365161lb # [ 98.676234] envoy[897]: conn peer=[2001:db8:1::1]:45848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=895162lb # [ 98.681014] envoy[897]: conn peer=[2001:db8:1::1]:45860 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4364 tx=2389 ms=16145163lb # [ 98.686909] envoy[897]: conn peer=[2001:db8:1::1]:45872 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=965164worker1 # [ 98.927216] nix-daemon[835]: reaped child process 1074, status = succeeded5165worker1 # [ 99.013148] nix-daemon[835]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5166worker1 # [ 99.031639] nix-daemon[835]: reaped child process 1093, status = succeeded5167lb # [ 99.013783] niks3-server[1028]: 2026/09/23 04:44:48 INFO Received uploads request method=POST path=/api/pending_closures5168worker1 # [ 99.050788] nix-grpc-daemon[861]: time=2026-09-23T04:44:48.624Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5169lb # [ 99.047697] niks3-server[1028]: 2026/09/23 04:44:48 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5170worker1 # [ 99.090945] nix-grpc-daemon[861]: time=2026-09-23T04:44:48.666Z level=INFO msg="Uploading 1 narinfos"5171lb # [ 99.059988] niks3-server[1028]: 2026/09/23 04:44:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5172lb # [ 99.070552] niks3-server[1028]: 2026/09/23 04:44:48 INFO Signed narinfos id=30 count=15173lb # [ 99.077602] niks3-server[1028]: 2026/09/23 04:44:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5174worker1 # [ 99.115071] nix-grpc-daemon[861]: time=2026-09-23T04:44:48.690Z level=INFO msg="Upload complete. (270ms)"5175lb # [ 99.089624] niks3-server[1028]: 2026/09/23 04:44:48 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5176lb # [ 99.098578] niks3-server[1028]: 2026/09/23 04:44:48 INFO Completed upload id=305177lb # [ 99.103020] niks3-server[1028]: 2026/09/23 04:44:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5178worker1 # [ 99.128072] nix-grpc-daemon[767]: ts=2026-09-23T04:44:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15179worker1 # [ 99.143299] nix-daemon[835]: reaped child process 1072, status = succeeded5180client # copying 1 paths...5181worker2 # [ 99.137961] nix-grpc-daemon[771]: ts=2026-09-23T04:44:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:429565182worker2 # [ 99.145913] nix-grpc-daemon[771]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...5183client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5184worker2 # [ 99.194283] nix-grpc-daemon[771]: ts=2026-09-23T04:44:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:429565185worker2 # [ 99.201432] nix-grpc-daemon[771]: ts=2026-09-23T04:44:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_out=1205186client # [ 99.334606] nix-daemon[1149]: reaped child process 1235, status = succeeded5187client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.89 seconds)5188client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25189client # [ 99.516062] nix-daemon[1149]: accepted connection from pid 1254, user root (trusted)5190client # this derivation will be built:5191client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5192worker2 # [ 99.657470] nix-grpc-daemon[771]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:429565193client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5194worker2 # [ 99.665181] nix-grpc-daemon[771]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429565195worker2 # [ 99.673313] nix-grpc-daemon[874]: time=2026-09-23T04:44:49.238Z level=INFO msg="All 1 paths already cached"5196worker2 # [ 99.686819] nix-grpc-daemon[771]: ts=2026-09-23T04:44:49Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05197worker2 # [ 99.694024] nix-grpc-daemon[771]: ts=2026-09-23T04:44:49Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15198worker2 # [ 99.731180] nix-grpc-daemon[771]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429565199worker2 # [ 99.746318] nix-grpc-daemon[771]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429565200client # copying 1 paths to 'lb:50051'...5201client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5202worker2 # [ 99.766992] nix-grpc-daemon[771]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429565203worker2 # [ 99.781916] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5204worker2 # [ 99.803858] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)5205worker2 # [ 99.894970] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5)5206worker2 # [ 99.990884] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 148 ms (attempt 3/5)5207worker2 # [ 100.143097] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 108 ms (attempt 4/5)5208worker2 # [ 100.260790] nix-grpc-daemon[771]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5209worker2 # [ 100.268284] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5210worker2 # [ 100.276143] nix-grpc-daemon[771]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5211worker2 # [ 100.437892] nix-daemon[859]: accepted connection from pid 1273, user nix-grpc-daemon (trusted)5212worker2 # [ 100.459011] nix-daemon[859]: reaped child process 1276, status = succeeded5213lb # [ 100.480855] niks3-server[1028]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures5214worker2 # [ 100.474238] nix-grpc-daemon[874]: time=2026-09-23T04:44:50.041Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5215worker2 # [ 100.475924] nix-grpc-daemon[874]: time=2026-09-23T04:44:50.042Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5216worker2 # [ 100.502608] nix-grpc-daemon[874]: time=2026-09-23T04:44:50.070Z level=INFO msg="Uploading 1 narinfos"5217lb # [ 100.509940] niks3-server[1028]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5218lb # [ 100.523684] niks3-server[1028]: 2026/09/23 04:44:50 INFO Signed narinfos id=31 count=15219lb # [ 100.534405] niks3-server[1028]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5220lb # [ 100.546951] niks3-server[1028]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5221worker2 # [ 100.537394] nix-grpc-daemon[874]: time=2026-09-23T04:44:50.105Z level=INFO msg="Upload complete. (248ms)"5222lb # [ 100.555758] niks3-server[1028]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5223lb # [ 100.563373] niks3-server[1028]: 2026/09/23 04:44:50 INFO Completed upload id=315224worker2 # [ 100.548294] nix-grpc-daemon[771]: ts=2026-09-23T04:44:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_in=5685225lb # [ 100.570739] niks3-server[1028]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5226worker2 # [ 100.560322] nix-grpc-daemon[771]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429565227worker2 # [ 100.573635] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5228worker2 # [ 100.578797] nix-daemon[859]: reaped child process 1266, status = succeeded5229worker2 # [ 100.590667] nix-daemon[859]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5230client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5231client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5232client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5233worker2 # [ 101.298416] nix-daemon[859]: reaped child process 1280, status = succeeded5234worker2 # [ 101.371017] nix-daemon[859]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5235worker2 # [ 101.390935] nix-daemon[859]: reaped child process 1299, status = succeeded5236lb # [ 101.408884] niks3-server[1028]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures5237worker2 # [ 101.404408] nix-grpc-daemon[874]: time=2026-09-23T04:44:50.970Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5238lb # [ 101.440500] niks3-server[1028]: 2026/09/23 04:44:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5239worker2 # [ 101.433525] nix-grpc-daemon[874]: time=2026-09-23T04:44:51.001Z level=INFO msg="Uploading 1 narinfos"5240lb # [ 101.455813] niks3-server[1028]: 2026/09/23 04:44:51 INFO Signed narinfos id=32 count=15241lb # [ 101.462770] niks3-server[1028]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5242lb # [ 101.474173] niks3-server[1028]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5243lb # [ 101.478554] niks3-server[1028]: 2026/09/23 04:44:51 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5244worker2 # [ 101.466321] nix-grpc-daemon[874]: time=2026-09-23T04:44:51.034Z level=INFO msg="Upload complete. (264ms)"5245lb # [ 101.485086] niks3-server[1028]: 2026/09/23 04:44:51 INFO Completed upload id=325246lb # [ 101.491396] niks3-server[1028]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5247client # copying 1 paths...5248worker2 # [ 101.476890] nix-grpc-daemon[771]: ts=2026-09-23T04:44:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15249client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5250worker2 # [ 101.495382] nix-grpc-daemon[771]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:429565251worker2 # [ 101.499525] nix-daemon[859]: reaped child process 1278, status = succeeded5252worker1 # [ 101.559328] nix-grpc-daemon[767]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:385505253worker1 # [ 101.572529] nix-grpc-daemon[767]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5254worker1 # [ 101.589294] nix-grpc-daemon[767]: ts=2026-09-23T04:44:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_out=1205255client # [ 101.678721] nix-daemon[1149]: reaped child process 1257, status = succeeded5256client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.34 seconds)5257client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25258client # [ 101.853742] nix-daemon[1149]: accepted connection from pid 1276, user root (trusted)5259client # this derivation will be built:5260client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5261worker2 # [ 101.993500] nix-grpc-daemon[771]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:429565262client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5263worker2 # [ 102.001384] nix-grpc-daemon[771]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429565264worker2 # [ 102.008689] nix-grpc-daemon[874]: time=2026-09-23T04:44:51.575Z level=INFO msg="All 1 paths already cached"5265worker2 # [ 102.020265] nix-grpc-daemon[771]: ts=2026-09-23T04:44:51Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05266worker2 # [ 102.026513] nix-grpc-daemon[771]: ts=2026-09-23T04:44:51Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15267worker1 # [ 102.113255] nix-grpc-daemon[767]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385505268worker1 # [ 102.128840] nix-grpc-daemon[767]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385505269client # copying 1 paths to 'lb:50051'...5270worker1 # [ 102.145714] nix-grpc-daemon[861]: time=2026-09-23T04:44:51.719Z level=INFO msg="All 1 paths already cached"5271client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5272worker1 # [ 102.151149] nix-grpc-daemon[767]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385505273worker1 # [ 102.166063] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5274worker1 # [ 102.325919] nix-daemon[835]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)5275worker1 # [ 102.344424] nix-daemon[835]: reaped child process 1101, status = succeeded5276lb # [ 102.323588] niks3-server[1028]: 2026/09/23 04:44:51 INFO Received uploads request method=POST path=/api/pending_closures5277worker1 # [ 102.357779] nix-grpc-daemon[861]: time=2026-09-23T04:44:51.933Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5278worker1 # [ 102.359316] nix-grpc-daemon[861]: time=2026-09-23T04:44:51.933Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5279worker1 # [ 102.386227] nix-grpc-daemon[861]: time=2026-09-23T04:44:51.961Z level=INFO msg="Uploading 1 narinfos"5280lb # [ 102.353719] niks3-server[1028]: 2026/09/23 04:44:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5281lb # [ 102.373516] niks3-server[1028]: 2026/09/23 04:44:51 INFO Signed narinfos id=33 count=15282lb # [ 102.381379] niks3-server[1028]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5283worker1 # [ 102.420938] nix-grpc-daemon[861]: time=2026-09-23T04:44:51.996Z level=INFO msg="Upload complete. (237ms)"5284lb # [ 102.398021] niks3-server[1028]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5285lb # [ 102.407343] niks3-server[1028]: 2026/09/23 04:44:51 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5286worker1 # [ 102.433100] nix-grpc-daemon[767]: ts=2026-09-23T04:44:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=0 paths=1 nar_bytes_in=5685287lb # [ 102.412408] niks3-server[1028]: 2026/09/23 04:44:51 INFO Completed upload id=335288lb # [ 102.414916] niks3-server[1028]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5289worker1 # [ 102.448110] nix-grpc-daemon[767]: ts=2026-09-23T04:44:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385505290worker1 # [ 102.454054] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5291worker1 # [ 102.458855] nix-daemon[835]: reaped child process 1096, status = succeeded5292client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5293worker1 # [ 102.471508] nix-daemon[835]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5294client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5295client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5296worker1 # [ 103.914522] nix-daemon[835]: reaped child process 1105, status = succeeded5297worker1 # [ 104.014809] nix-daemon[835]: accepted connection from pid 1121, user nix-grpc-daemon (trusted)5298worker1 # [ 104.033475] nix-daemon[835]: reaped child process 1124, status = succeeded5299lb # [ 104.014673] niks3-server[1028]: 2026/09/23 04:44:53 INFO Received uploads request method=POST path=/api/pending_closures5300worker1 # [ 104.050758] nix-grpc-daemon[861]: time=2026-09-23T04:44:53.624Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5301lb # [ 104.046197] niks3-server[1028]: 2026/09/23 04:44:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5302worker1 # [ 104.080256] nix-grpc-daemon[861]: time=2026-09-23T04:44:53.655Z level=INFO msg="Uploading 1 narinfos"5303lb # [ 104.059783] niks3-server[1028]: 2026/09/23 04:44:53 INFO Signed narinfos id=34 count=15304lb # [ 104.070142] niks3-server[1028]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5305lb # [ 104.081267] niks3-server[1028]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5306worker1 # [ 104.114333] nix-grpc-daemon[861]: time=2026-09-23T04:44:53.690Z level=INFO msg="Upload complete. (233ms)"5307lb # [ 104.092096] niks3-server[1028]: 2026/09/23 04:44:53 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5308lb # [ 104.097847] niks3-server[1028]: 2026/09/23 04:44:53 INFO Completed upload id=345309lb # [ 104.101666] niks3-server[1028]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5310client # copying 1 paths...5311worker1 # [ 104.124956] nix-grpc-daemon[767]: ts=2026-09-23T04:44:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38550 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15312worker1 # [ 104.140992] nix-grpc-daemon[767]: ts=2026-09-23T04:44:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:385505313worker1 # [ 104.145966] nix-daemon[835]: reaped child process 1103, status = succeeded5314client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5315worker2 # [ 104.132930] nix-grpc-daemon[771]: ts=2026-09-23T04:44:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:429565316worker2 # [ 104.145349] nix-grpc-daemon[771]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5317worker2 # [ 104.163901] nix-grpc-daemon[771]: ts=2026-09-23T04:44:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42956 duration_s=0 paths=1 nar_bytes_out=1205318client # [ 104.297756] nix-daemon[1149]: reaped child process 1279, status = succeeded5319client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.62 seconds)5320(finished: subtest: an input only one worker has still reaches the builder, in 8.48 seconds)5321subtest: upload whose reference the worker lacks is completed from the cache5322client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5323client # this derivation will be built:5324client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5325client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5326client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5327client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5328client # this derivation will be built:5329client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5330client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5331client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)5332client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5333client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5334worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5335worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5336worker2 # [ 105.295465] systemd[1]: Stopping Nix worker-protocol over gRPC...5337worker2 # [ 105.970270] nix-grpc-daemon[771]: ts=2026-09-23T04:44:55Z level=info event=signal_exit5338lb # [ 106.889329] niks3-server[1028]: 2026/09/23 04:44:56 INFO lead: released remote=192.168.1.4:504985339worker1 # [ 107.913097] nix-grpc-daemon[767]: ts=2026-09-23T04:44:57Z level=info event=scheduler_restarting addr=lb:500515340worker2 # [ 107.878364] nix-grpc-daemon[771]: ts=2026-09-23T04:44:57Z level=info event=scheduler_restarting5341worker2 # [ 107.882587] nix-grpc-daemon[771]: ts=2026-09-23T04:44:57Z level=info event=scheduler_restarting addr=lb:500515342worker2 # [ 107.887316] nix-grpc-daemon[771]: ts=2026-09-23T04:44:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:429465343worker2 # [ 107.892432] nix-grpc-daemon[771]: ts=2026-09-23T04:44:57Z level=info event=worker_gone addr=192.168.1.3:500515344worker2 # [ 107.897192] nix-grpc-daemon[771]: ts=2026-09-23T04:44:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:429465345worker2 # [ 107.901835] nix-grpc-daemon[771]: ts=2026-09-23T04:44:57Z level=info event=worker_gone addr=192.168.1.4:500515346worker2 # [ 107.928888] nix-daemon[859]: reaped child process 861, status = succeeded5347worker1 # [ 108.112970] nix-grpc-daemon[767]: ts=2026-09-23T04:44:57Z level=info event=scheduler_connected addr=lb:500515348worker1 # [ 108.119060] nix-grpc-daemon[767]: ts=2026-09-23T04:44:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5349worker2 # [ 108.095684] nix-grpc-daemon[771]: ts=2026-09-23T04:44:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5350worker2 # [ 108.163224] nix-grpc-daemon[771]: E0000 00:00:1790138697.730778 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5351worker2 # [ 108.169830] nix-grpc-daemon[771]: E0000 00:00:1790138697.731170 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5352worker1 # [ 108.319881] nix-grpc-daemon[767]: ts=2026-09-23T04:44:57Z level=info event=scheduler_connected addr=lb:500515353worker1 # [ 108.325253] nix-grpc-daemon[767]: ts=2026-09-23T04:44:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5354worker1 # [ 108.526088] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_connected addr=lb:500515355worker1 # [ 108.531634] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5356lb # [ 108.658331] envoy[897]: conn peer=[2001:db8:1::1]:45886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4270 tx=2050 ms=24795357lb # [ 108.663431] envoy[897]: conn peer=[2001:db8:1::1]:45894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=715358lb # [ 108.668564] envoy[897]: conn peer=[2001:db8:1::1]:45910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3728 tx=2125 ms=19545359lb # [ 108.673466] envoy[897]: conn peer=[2001:db8:1::1]:44606 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=965360lb # [ 108.678414] envoy[897]: conn peer=[2001:db8:1::1]:44612 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2098 ms=22315361lb # [ 108.684377] envoy[897]: conn peer=[2001:db8:1::1]:44618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=965362lb # [ 108.689073] envoy[897]: conn peer=[2001:db8:1::4]:48078 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=84578 tx=4720 ms=890845363worker1 # [ 108.732750] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_connected addr=lb:500515364worker1 # [ 108.738002] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5365worker1 # [ 108.940760] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_connected addr=lb:500515366worker1 # [ 108.945063] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367worker2 # [ 109.017430] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5368worker2 # [ 109.024885] systemd[1]: Stopped Nix worker-protocol over gRPC.5369worker2 # [ 109.027488] systemd[1]: nix-grpc-daemon.service: Consumed 3.973s CPU time over 1min 30.258s wall clock time, 69.7M memory peak, 122K incoming IP traffic, 315.2K outgoing IP traffic.5370worker2 # [ 109.044864] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5371worker2 # [ 109.046920] systemd[1]: Closed Nix worker-protocol over gRPC.5372worker2 # [ 109.051428] systemd[1]: nix-grpc-daemon.socket: Consumed 601us CPU time over 1min 38.768s wall clock time, 256K memory peak, 129.6K incoming IP traffic, 108.7K outgoing IP traffic.5373lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5374lb: must succeed: curl -sf localhost:9901/clusters5375worker1 # [ 109.141807] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_connected addr=lb:500515376worker1 # [ 109.147319] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5378lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5379client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25380worker1 # [ 109.347995] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_connected addr=lb:500515381worker1 # [ 109.353245] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5382worker1 # [ 109.388261] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:385505383worker1 # [ 109.396922] nix-grpc-daemon[767]: ts=2026-09-23T04:44:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:385505384client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5385client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5386worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5387worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5388worker2 # [ 109.477990] systemd[1]: Listening on Nix worker-protocol over gRPC.5389worker1 # [ 109.554245] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_connected addr=lb:500515390worker1 # [ 109.564599] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5391worker1 # [ 109.602038] systemd[1]: Stopping Nix worker-protocol over gRPC...5392worker1 # [ 109.762403] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_connected addr=lb:500515393worker1 # [ 109.767768] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5394worker1 # [ 109.956776] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=signal_exit5395worker1 # [ 109.968772] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_connected addr=lb:500515396worker1 # [ 109.974885] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5397worker2 # [ 110.015247] systemd[1]: Starting scheduler-token.service...5398worker2 # [ 110.107067] systemd[1]: scheduler-token.service: Deactivated successfully.5399worker2 # [ 110.117052] systemd[1]: Finished scheduler-token.service.5400worker2 # [ 110.119206] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 96ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5401worker1 # [ 110.175491] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_connected addr=lb:500515402worker2 # [ 110.140696] systemd[1]: Starting Nix worker-protocol over gRPC...5403worker1 # [ 110.180746] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5404worker2 # [ 110.264438] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5405worker2 # [ 110.267102] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5406worker2 # [ 110.270997] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15407worker2 # [ 110.276064] nix-grpc-daemon[1326]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5408worker2 # [ 110.277499] nix-grpc-daemon[1326]: E0000 00:00:1790138699.843749 1326 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5409lb # [ 110.317103] niks3-server[1028]: 2026/09/23 04:44:59 INFO lead: acquired remote=192.168.1.4:530445410worker2 # [ 110.291612] nix-grpc-daemon[1326]: E0000 00:00:1790138699.850054 1326 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5411worker2 # [ 110.324224] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5412worker2 # [ 110.325892] systemd[1]: Started Nix worker-protocol over gRPC.5413worker2 # [ 110.326691] nix-grpc-daemon[1326]: E0000 00:00:1790138699.863334 1369 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5414worker2 # [ 110.335052] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=scheduler_take_over5415worker2 # [ 110.336782] nix-grpc-daemon[1326]: E0000 00:00:1790138699.882616 1369 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5416worker2 # [ 110.339165] nix-grpc-daemon[1326]: E0000 00:00:1790138699.882757 1369 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5417worker1 # [ 110.381420] nix-grpc-daemon[767]: ts=2026-09-23T04:44:59Z level=info event=scheduler_connected addr=lb:500515418worker2 # [ 110.345222] nix-grpc-daemon[1326]: E0000 00:00:1790138699.902388 1369 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5419worker2 # [ 110.350923] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:448545420worker2 # [ 110.352872] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05421worker2 # [ 110.355110] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=scheduler_connected addr=lb:500515422worker2 # [ 110.359249] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=oidc_jwks provider=mock keys=15423worker2 # [ 110.360770] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:448545424worker2 # [ 110.362468] nix-grpc-daemon[1326]: ts=2026-09-23T04:44:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05425worker2 # [ 111.877017] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448545426worker2 # [ 111.881728] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:01Z level=info event=worker_gone addr=192.168.1.3:500515427worker1 # [ 111.940707] nix-grpc-daemon[767]: ts=2026-09-23T04:45:01Z level=info event=scheduler_restarting5428worker1 # [ 111.950488] nix-grpc-daemon[767]: ts=2026-09-23T04:45:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5429worker1 # [ 111.969190] nix-daemon[835]: reaped child process 837, status = succeeded5430worker1 # [ 112.104754] nix-grpc-daemon[767]: E0000 00:00:1790138701.680347 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5431worker1 # [ 112.129838] nix-grpc-daemon[767]: E0000 00:00:1790138701.705576 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5432worker1 # [ 112.995278] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5433worker1 # [ 113.002804] systemd[1]: Stopped Nix worker-protocol over gRPC.5434worker1 # [ 113.009223] systemd[1]: nix-grpc-daemon.service: Consumed 2.807s CPU time over 1min 34.316s wall clock time, 60.3M memory peak, 90.3K incoming IP traffic, 163.9K outgoing IP traffic.5435worker1 # [ 113.023489] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5436worker1 # [ 113.032350] systemd[1]: Closed Nix worker-protocol over gRPC.5437worker1 # [ 113.034294] systemd[1]: nix-grpc-daemon.socket: Consumed 653us CPU time over 1min 42.674s wall clock time, 256K memory peak, 95.8K incoming IP traffic, 85.3K outgoing IP traffic.5438lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5439lb: must succeed: curl -sf localhost:9901/clusters5440lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5441lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5442client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25443worker2 # [ 113.289194] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427945444worker2 # [ 113.303672] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5445worker2 # [ 113.333894] nix-grpc-daemon[1326]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)5446worker2 # [ 113.377835] nix-grpc-daemon[1326]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 2/5)5447worker2 # [ 113.519776] nix-grpc-daemon[1326]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 323 ms (attempt 3/5)5448worker2 # [ 113.845021] nix-grpc-daemon[1326]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 4/5)5449worker2 # [ 113.983212] nix-grpc-daemon[1326]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5450worker2 # [ 113.990043] nix-grpc-daemon[1326]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5451client # copying 2 paths...5452client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5453worker2 # [ 114.013658] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427945454client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5455worker2 # [ 114.027105] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5456worker2 # [ 114.062506] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:03Z level=info event=niks3_push_started pid=13875457worker2 # [ 114.217204] nix-grpc-daemon[1387]: time=2026-09-23T04:45:03.783Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5458worker2 # [ 114.325623] nix-daemon[859]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5459worker2 # [ 114.341720] nix-daemon[859]: reaped child process 1402, status = succeeded5460lb # [ 114.367651] niks3-server[1028]: 2026/09/23 04:45:03 INFO Received uploads request method=POST path=/api/pending_closures5461lb # [ 114.377982] niks3-server[1028]: 2026/09/23 04:45:03 INFO Received uploads request method=POST path=/api/pending_closures5462worker2 # [ 114.372066] nix-grpc-daemon[1387]: time=2026-09-23T04:45:03.938Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5463worker2 # [ 114.373547] nix-grpc-daemon[1387]: time=2026-09-23T04:45:03.938Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5464worker2 # [ 114.375247] nix-grpc-daemon[1387]: time=2026-09-23T04:45:03.938Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5465lb # [ 114.439314] niks3-server[1028]: 2026/09/23 04:45:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5466worker2 # [ 114.439135] nix-grpc-daemon[1387]: time=2026-09-23T04:45:04.005Z level=INFO msg="Uploading 3 narinfos"5467lb # [ 114.457649] niks3-server[1028]: 2026/09/23 04:45:04 INFO Signed narinfos id=36 count=25468lb # [ 114.459634] niks3-server[1028]: 2026/09/23 04:45:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5469lb # [ 114.472926] niks3-server[1028]: 2026/09/23 04:45:04 INFO Signed narinfos id=35 count=15470lb # [ 114.474801] niks3-server[1028]: 2026/09/23 04:45:04 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5471lb # [ 114.489665] niks3-server[1028]: 2026/09/23 04:45:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5472lb # [ 114.498346] niks3-server[1028]: 2026/09/23 04:45:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5473lb # [ 114.504257] niks3-server[1028]: 2026/09/23 04:45:04 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5474lb # [ 114.514766] niks3-server[1028]: 2026/09/23 04:45:04 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5475lb # [ 114.519731] niks3-server[1028]: 2026/09/23 04:45:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5476lb # [ 114.524277] niks3-server[1028]: 2026/09/23 04:45:04 INFO Completed upload id=355477lb # [ 114.525979] niks3-server[1028]: 2026/09/23 04:45:04 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5478lb # [ 114.529182] niks3-server[1028]: 2026/09/23 04:45:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5479lb # [ 114.531455] niks3-server[1028]: 2026/09/23 04:45:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5480lb # [ 114.535364] niks3-server[1028]: 2026/09/23 04:45:04 INFO Completed upload id=365481worker2 # [ 114.519007] nix-grpc-daemon[1387]: time=2026-09-23T04:45:04.086Z level=INFO msg="Upload complete. (302ms)"5482worker2 # [ 114.531306] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42794 duration_s=0 paths=2 nar_bytes_in=3125483worker2 # [ 114.549786] nix-daemon[859]: reaped child process 1385, status = succeeded5484client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.49 seconds)5485worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5486worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5487worker1 # [ 114.711131] systemd[1]: Listening on Nix worker-protocol over gRPC.5488worker1 # [ 114.720180] systemd[1]: Starting Nix worker-protocol over gRPC...5489worker1 # [ 114.868468] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5490worker1 # [ 114.871191] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5491worker1 # [ 114.875027] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15492worker1 # [ 114.880052] nix-grpc-daemon[1147]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5493worker1 # [ 114.881491] nix-grpc-daemon[1147]: E0000 00:00:1790138704.455467 1147 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5494lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5495lb: must succeed: curl -sf localhost:9901/clusters5496worker1 # [ 114.927159] nix-grpc-daemon[1147]: E0000 00:00:1790138704.461753 1147 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5497worker1 # [ 114.930223] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5498worker1 # [ 114.932080] nix-grpc-daemon[1147]: E0000 00:00:1790138704.467955 1187 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5499worker1 # [ 114.934418] nix-grpc-daemon[1147]: E0000 00:00:1790138704.485708 1187 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5500worker1 # [ 114.936815] nix-grpc-daemon[1147]: E0000 00:00:1790138704.485864 1187 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5501worker1 # [ 114.939152] nix-grpc-daemon[1147]: E0000 00:00:1790138704.502306 1187 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5502worker1 # [ 114.944756] systemd[1]: Started Nix worker-protocol over gRPC.5503worker1 # [ 114.947318] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:04Z level=info event=scheduler_connected addr=lb:500515504worker2 # [ 114.907311] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:04Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:448545505worker2 # [ 114.913411] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05506lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5507lb: must succeed: curl -sf localhost:9901/clusters5508lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5509lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5510(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.72 seconds)5511subtest: a niks3 restart does not cost the leader its role5512lb: must succeed: curl -sf localhost:9901/clusters5513lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5514lb: must succeed: curl -sf localhost:9901/clusters5515lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5516lb # [ 116.164718] systemd[1]: Stopping niks3 server...5517lb # [ 116.165456] niks3-server[1028]: 2026/09/23 04:45:05 INFO Shutdown signal received, draining in-flight requests timeout=10s5518lb # [ 116.171339] niks3-server[1028]: 2026/09/23 04:45:05 INFO lead: released remote=192.168.1.4:530445519worker1 # [ 116.195919] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:05Z level=info event=scheduler_lock_lost status=2005520worker2 # [ 116.154342] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:05Z level=info event=scheduler_lock_lost status=2005521lb # [ 116.191183] systemd[1]: niks3.service: Deactivated successfully.5522lb # [ 116.193083] systemd[1]: Stopped niks3 server.5523lb # [ 116.194642] systemd[1]: niks3.service: Consumed 650ms CPU time over 1min 25.543s wall clock time, 27.5M memory peak, 95K incoming IP traffic, 138.5K outgoing IP traffic.5524lb # [ 116.204446] systemd[1]: Starting niks3 server...5525lb # [ 116.268484] niks3-server[1166]: 2026/09/23 04:45:05 goose: no migrations to run. current version: 202609200000005526lb # [ 116.274590] niks3-server[1166]: 2026/09/23 04:45:05 OK 1_commit_pending_closure.sql (7.48ms)5527lb # [ 116.278220] niks3-server[1166]: 2026/09/23 04:45:05 OK 2_object_stats_trigger.sql (1.97ms)5528lb # [ 116.279381] niks3-server[1166]: 2026/09/23 04:45:05 goose: up to current file version: 25529lb # [ 116.281347] niks3-server[1166]: 2026/09/23 04:45:05 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5530lb # [ 116.292293] niks3-server[1166]: 2026/09/23 04:45:05 INFO Read proxy enabled — serving cache objects from S35531lb # [ 116.293629] niks3-server[1166]: 2026/09/23 04:45:05 INFO Using socket-activated listener address=0.0.0.0:57515532lb # [ 116.294892] niks3-server[1166]: 2026/09/23 04:45:05 INFO systemd watchdog enabled interval=15s5533lb # [ 116.299383] systemd[1]: Started niks3 server.5534lb # [ 116.300384] niks3-server[1166]: 2026/09/23 04:45:05 INFO Starting HTTP server address=0.0.0.0:57515535lb: waiting for unit niks3.service5536lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5537lb: must succeed: curl -sf localhost:9901/clusters5538lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5539worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5540worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5541lb: must succeed: curl -sf localhost:9901/clusters5542lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5543lb: must succeed: curl -sf localhost:9901/clusters5544lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5545worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5546worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5547client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25548worker1 # [ 116.805732] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577205549worker1 # [ 116.820199] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5550client # don't know how to build these paths:5551client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5552worker2 # [ 116.801818] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427945553worker2 # [ 116.833827] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:06Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05554worker2 # [ 116.841466] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:06Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15555worker2 # [ 116.879503] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427945556worker2 # [ 116.895325] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427945557client # copying 1 paths to 'lb:50051'...5558client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5559worker2 # [ 116.911062] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427945560worker2 # [ 116.926041] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5561worker2 # [ 117.086332] nix-daemon[859]: accepted connection from pid 1419, user nix-grpc-daemon (trusted)5562worker2 # [ 117.105051] nix-daemon[859]: reaped child process 1422, status = succeeded5563lb # [ 117.126878] niks3-server[1166]: 2026/09/23 04:45:06 INFO Received uploads request method=POST path=/api/pending_closures5564worker2 # [ 117.123783] nix-grpc-daemon[1387]: time=2026-09-23T04:45:06.690Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5565worker2 # [ 117.125596] nix-grpc-daemon[1387]: time=2026-09-23T04:45:06.690Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5566lb # [ 117.169023] niks3-server[1166]: 2026/09/23 04:45:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5567worker2 # [ 117.188157] nix-grpc-daemon[1387]: time=2026-09-23T04:45:06.755Z level=INFO msg="Uploading 1 narinfos"5568lb # [ 117.206525] niks3-server[1166]: 2026/09/23 04:45:06 INFO Signed narinfos id=37 count=15569lb # [ 117.223482] niks3-server[1166]: 2026/09/23 04:45:06 INFO lead: acquired remote=192.168.1.4:449905570lb # [ 117.232344] niks3-server[1166]: 2026/09/23 04:45:06 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5571lb # [ 117.248333] niks3-server[1166]: 2026/09/23 04:45:06 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5572lb # [ 117.258778] niks3-server[1166]: 2026/09/23 04:45:06 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5573lb # [ 117.265357] niks3-server[1166]: 2026/09/23 04:45:06 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5574lb # [ 117.278821] niks3-server[1166]: 2026/09/23 04:45:06 INFO Completed upload id=375575worker2 # [ 117.261826] nix-grpc-daemon[1387]: time=2026-09-23T04:45:06.829Z level=INFO msg="Upload complete. (319ms)"5576worker2 # [ 117.273164] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42794 duration_s=0 paths=1 nar_bytes_in=5445577worker2 # [ 117.293350] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427945578worker2 # [ 117.303364] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5579worker2 # [ 117.311052] nix-daemon[859]: reaped child process 1417, status = succeeded5580client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5581worker2 # [ 117.327188] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5582client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5583client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5584client # farm-a> LOG-farm-a-after-niks3-restart5585worker2 # [ 118.416924] nix-daemon[859]: reaped child process 1428, status = succeeded5586worker2 # [ 118.519726] nix-daemon[859]: accepted connection from pid 1444, user nix-grpc-daemon (trusted)5587worker2 # [ 118.537546] nix-daemon[859]: reaped child process 1447, status = succeeded5588lb # [ 118.566472] niks3-server[1166]: 2026/09/23 04:45:08 INFO Received uploads request method=POST path=/api/pending_closures5589worker2 # [ 118.568533] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5590worker2 # [ 118.570424] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.134Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5591lb # [ 118.620553] niks3-server[1166]: 2026/09/23 04:45:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5592worker2 # [ 118.613379] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.181Z level=INFO msg="Uploading 1 narinfos"5593lb # [ 118.635932] niks3-server[1166]: 2026/09/23 04:45:08 INFO Signed narinfos id=38 count=15594lb # [ 118.645677] niks3-server[1166]: 2026/09/23 04:45:08 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5595lb # [ 118.655005] niks3-server[1166]: 2026/09/23 04:45:08 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5596lb # [ 118.663111] niks3-server[1166]: 2026/09/23 04:45:08 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5597worker2 # [ 118.660335] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.228Z level=INFO msg="Upload complete. (316ms)"5598lb # [ 118.677194] niks3-server[1166]: 2026/09/23 04:45:08 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5599lb # [ 118.681919] niks3-server[1166]: 2026/09/23 04:45:08 INFO Completed upload id=385600lb # [ 118.689154] envoy[897]: conn peer=[2001:db8:1::1]:44626 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=295601lb # [ 118.692905] envoy[897]: conn peer=[2001:db8:1::3]:53634 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=18207 tx=4887 ms=929465602lb # [ 118.696513] envoy[897]: conn peer=[2001:db8:1::1]:34966 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=282 ms=12585603worker2 # [ 118.680982] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42794 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15604lb # [ 118.703239] niks3-server[1166]: 2026/09/23 04:45:08 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5605worker2 # [ 118.703392] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:08Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05606worker2 # [ 118.708745] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:08Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15607worker2 # [ 118.713945] nix-daemon[859]: reaped child process 1426, status = succeeded5608worker2 # [ 118.739753] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427945609worker2 # [ 118.755676] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427945610worker2 # [ 118.771105] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.339Z level=INFO msg="All 1 paths already cached"5611client # copying 1 paths to 'lb:50051'...5612client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5613worker2 # [ 118.782168] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427945614worker2 # [ 118.796101] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5615worker2 # [ 118.945402] nix-daemon[859]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)5616worker2 # [ 118.962121] nix-daemon[859]: reaped child process 1456, status = succeeded5617lb # [ 118.985784] niks3-server[1166]: 2026/09/23 04:45:08 INFO Received uploads request method=POST path=/api/pending_closures5618worker2 # [ 118.982829] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.550Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5619worker2 # [ 118.984674] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.550Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5620lb # [ 119.020665] niks3-server[1166]: 2026/09/23 04:45:08 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5621worker2 # [ 119.015466] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.583Z level=INFO msg="Uploading 1 narinfos"5622lb # [ 119.034277] niks3-server[1166]: 2026/09/23 04:45:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5623lb # [ 119.046710] niks3-server[1166]: 2026/09/23 04:45:08 INFO Signed narinfos id=39 count=15624lb # [ 119.052918] niks3-server[1166]: 2026/09/23 04:45:08 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5625lb # [ 119.066362] niks3-server[1166]: 2026/09/23 04:45:08 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5626worker2 # [ 119.049524] nix-grpc-daemon[1387]: time=2026-09-23T04:45:08.617Z level=INFO msg="Upload complete. (237ms)"5627lb # [ 119.071908] niks3-server[1166]: 2026/09/23 04:45:08 INFO Completed upload id=395628lb # [ 119.075763] niks3-server[1166]: 2026/09/23 04:45:08 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5629worker2 # [ 119.060333] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42794 duration_s=0 paths=1 nar_bytes_in=6565630worker2 # [ 119.081464] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427945631worker2 # [ 119.092593] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5632worker2 # [ 119.096172] nix-daemon[859]: reaped child process 1451, status = succeeded5633client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5634client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5635worker2 # [ 119.114022] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5636client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5637client # farm-b> LOG-farm-b-after-niks3-restart5638worker2 # [ 120.545160] nix-daemon[859]: reaped child process 1460, status = succeeded5639worker2 # [ 120.654809] nix-daemon[859]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)5640worker2 # [ 120.676272] nix-daemon[859]: reaped child process 1479, status = succeeded5641lb # [ 120.693084] niks3-server[1166]: 2026/09/23 04:45:10 INFO Received uploads request method=POST path=/api/pending_closures5642worker2 # [ 120.687823] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.254Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5643worker2 # [ 120.689571] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.254Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5644lb # [ 120.733603] niks3-server[1166]: 2026/09/23 04:45:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5645worker2 # [ 120.725772] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.293Z level=INFO msg="Uploading 1 narinfos"5646lb # [ 120.744948] niks3-server[1166]: 2026/09/23 04:45:10 INFO Signed narinfos id=40 count=15647lb # [ 120.756420] niks3-server[1166]: 2026/09/23 04:45:10 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5648lb # [ 120.768651] niks3-server[1166]: 2026/09/23 04:45:10 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5649worker2 # [ 120.757334] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.325Z level=INFO msg="Upload complete. (231ms)"5650lb # [ 120.776238] niks3-server[1166]: 2026/09/23 04:45:10 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5651lb # [ 120.778904] niks3-server[1166]: 2026/09/23 04:45:10 INFO Completed upload id=405652lb # [ 120.786615] niks3-server[1166]: 2026/09/23 04:45:10 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5653worker2 # [ 120.768284] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42794 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15654lb # [ 120.789813] niks3-server[1166]: 2026/09/23 04:45:10 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5655worker2 # [ 120.788792] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:10Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05656worker2 # [ 120.792387] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:10Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15657worker2 # [ 120.796750] nix-daemon[859]: reaped child process 1458, status = succeeded5658worker2 # [ 120.827659] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427945659worker2 # [ 120.842856] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427945660worker2 # [ 120.858191] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.426Z level=INFO msg="All 2 paths already cached"5661client # copying 1 paths to 'lb:50051'...5662client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5663worker2 # [ 120.869596] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427945664worker2 # [ 120.883706] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5665worker2 # [ 121.032817] nix-daemon[859]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)5666worker2 # [ 121.052896] nix-daemon[859]: reaped child process 1488, status = succeeded5667lb # [ 121.070755] niks3-server[1166]: 2026/09/23 04:45:10 INFO Received uploads request method=POST path=/api/pending_closures5668worker2 # [ 121.064127] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.631Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5669worker2 # [ 121.065781] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.631Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5670lb # [ 121.104670] niks3-server[1166]: 2026/09/23 04:45:10 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5671worker2 # [ 121.100909] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.668Z level=INFO msg="Uploading 1 narinfos"5672lb # [ 121.120445] niks3-server[1166]: 2026/09/23 04:45:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5673lb # [ 121.130543] niks3-server[1166]: 2026/09/23 04:45:10 INFO Signed narinfos id=41 count=15674lb # [ 121.138278] niks3-server[1166]: 2026/09/23 04:45:10 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5675worker2 # [ 121.129025] nix-grpc-daemon[1387]: time=2026-09-23T04:45:10.697Z level=INFO msg="Upload complete. (229ms)"5676lb # [ 121.146477] niks3-server[1166]: 2026/09/23 04:45:10 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5677lb # [ 121.152509] niks3-server[1166]: 2026/09/23 04:45:10 INFO Completed upload id=415678lb # [ 121.156334] niks3-server[1166]: 2026/09/23 04:45:10 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5679worker2 # [ 121.139630] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42794 duration_s=0 paths=1 nar_bytes_in=7845680worker2 # [ 121.153671] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427945681worker2 # [ 121.160433] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5682worker2 # [ 121.165384] nix-daemon[859]: reaped child process 1483, status = succeeded5683client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5684worker2 # [ 121.178416] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5685client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5686client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5687client # farm-top> LOG-farm-top-after-niks3-restart5688worker2 # [ 122.608481] nix-daemon[859]: reaped child process 1492, status = succeeded5689worker2 # [ 122.704890] nix-daemon[859]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)5690worker2 # [ 122.724276] nix-daemon[859]: reaped child process 1511, status = succeeded5691lb # [ 122.746661] niks3-server[1166]: 2026/09/23 04:45:12 INFO Received uploads request method=POST path=/api/pending_closures5692worker2 # [ 122.741706] nix-grpc-daemon[1387]: time=2026-09-23T04:45:12.308Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5693worker2 # [ 122.743176] nix-grpc-daemon[1387]: time=2026-09-23T04:45:12.308Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5694lb # [ 122.780393] niks3-server[1166]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5695lb # [ 122.802967] niks3-server[1166]: 2026/09/23 04:45:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5696worker2 # [ 122.791594] nix-grpc-daemon[1387]: time=2026-09-23T04:45:12.359Z level=INFO msg="Uploading 1 narinfos"5697lb # [ 122.812189] niks3-server[1166]: 2026/09/23 04:45:12 INFO Signed narinfos id=42 count=15698lb # [ 122.814528] niks3-server[1166]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5699lb # [ 122.825392] niks3-server[1166]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5700lb # [ 122.833560] niks3-server[1166]: 2026/09/23 04:45:12 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5701lb # [ 122.838092] niks3-server[1166]: 2026/09/23 04:45:12 INFO Completed upload id=425702worker2 # [ 122.821164] nix-grpc-daemon[1387]: time=2026-09-23T04:45:12.389Z level=INFO msg="Upload complete. (232ms)"5703lb # [ 122.839837] niks3-server[1166]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5704worker2 # [ 122.833166] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42794 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15705worker2 # [ 122.852084] nix-daemon[859]: reaped child process 1490, status = succeeded5706client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.33 seconds)5707client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5708client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5709(finished: subtest: a niks3 restart does not cost the leader its role, in 7.03 seconds)5710subtest: scheduler down: the other node takes the lock and keeps it5711lb: must succeed: curl -sf localhost:9901/clusters5712lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5713lb: must succeed: curl -sf localhost:9901/clusters5714lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5715lb: must succeed: curl -sf localhost:9901/clusters5716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5717lb: must succeed: curl -sf localhost:9901/clusters5718lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5719worker2 # [ 123.309214] systemd[1]: Stopping Nix worker-protocol over gRPC...5720worker2 # [ 124.298448] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:13Z level=info event=signal_exit5721lb # [ 125.190723] niks3-server[1166]: 2026/09/23 04:45:14 INFO lead: released remote=192.168.1.4:449905722worker1 # [ 126.211943] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:15Z level=info event=scheduler_restarting addr=lb:500515723worker2 # [ 126.174589] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:15Z level=info event=scheduler_restarting5724worker2 # [ 126.180062] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:15Z level=info event=scheduler_restarting addr=lb:500515725worker2 # [ 126.184749] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:15Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448545726worker2 # [ 126.190161] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:15Z level=info event=worker_gone addr=192.168.1.3:500515727worker2 # [ 126.195348] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:15Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448545728worker2 # [ 126.201312] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:15Z level=info event=worker_gone addr=192.168.1.4:500515729worker2 # [ 126.224217] nix-daemon[859]: reaped child process 1379, status = succeeded5730worker2 # [ 126.334172] nix-grpc-daemon[1326]: E0000 00:00:1790138715.901996 1372 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5731worker2 # [ 126.340887] nix-grpc-daemon[1326]: E0000 00:00:1790138715.902202 1372 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5732worker1 # [ 126.411628] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:15Z level=info event=scheduler_connected addr=lb:500515733worker1 # [ 126.417199] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5734worker2 # [ 126.390436] nix-grpc-daemon[1326]: ts=2026-09-23T04:45:15Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5735worker2 # [ 126.447189] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5736worker2 # [ 126.452138] systemd[1]: Stopped Nix worker-protocol over gRPC.5737worker2 # [ 126.459737] systemd[1]: nix-grpc-daemon.service: Consumed 1.329s CPU time over 16.323s wall clock time, 38.8M memory peak, 46.5K incoming IP traffic, 78.3K outgoing IP traffic.5738worker2 # [ 126.471642] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5739worker2 # [ 126.478021] systemd[1]: Closed Nix worker-protocol over gRPC.5740worker2 # [ 126.479703] systemd[1]: nix-grpc-daemon.socket: Consumed 807us CPU time over 17.006s wall clock time, 256K memory peak, 39K incoming IP traffic, 29.3K outgoing IP traffic.5741worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5742worker1 # [ 126.617667] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_connected addr=lb:500515743worker1 # [ 126.621543] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5744worker1 # [ 126.822395] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_connected addr=lb:500515745worker1 # [ 126.827616] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5746worker1 # [ 127.028277] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_connected addr=lb:500515747worker1 # [ 127.033296] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5748worker1 # [ 127.203090] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_take_over5749lb # [ 127.179269] niks3-server[1166]: 2026/09/23 04:45:16 INFO lead: acquired remote=192.168.1.3:558005750worker1 # [ 127.235720] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_connected addr=lb:500515751worker1 # [ 127.239869] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5752worker1 # [ 127.436456] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=scheduler_connected addr=lb:500515753worker1 # [ 127.441567] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker1 # [ 127.642397] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=scheduler_connected addr=lb:500515755worker1 # [ 127.652934] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5756worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.17 seconds)5757worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5758worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5759worker1 # [ 127.848798] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=scheduler_connected addr=lb:500515760worker1 # [ 127.854115] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5761worker1 # [ 128.054683] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=scheduler_connected addr=lb:500515762worker1 # [ 128.063737] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:523025763worker1 # [ 128.069241] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05764lb # [ 128.661000] envoy[897]: conn peer=[2001:db8:1::1]:34980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9766 tx=4412 ms=60865765lb # [ 128.661254] envoy[897]: conn peer=[2001:db8:1::4]:53910 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1708 tx=442 ms=160345766worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5767worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5768client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25769client # don't know how to build these paths:5770client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5771worker1 # [ 129.020274] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577205772worker1 # [ 129.026673] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577205773worker1 # [ 129.044976] nix-grpc-daemon[1147]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)5774worker1 # [ 129.075140] nix-grpc-daemon[1147]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 2/5)5775worker1 # [ 129.094992] nix-grpc-daemon[1147]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 3/5)5776worker1 # [ 129.169997] nix-grpc-daemon[1147]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 268 ms (attempt 4/5)5777worker1 # [ 129.443113] nix-grpc-daemon[1147]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5778worker1 # [ 129.448287] nix-grpc-daemon[1147]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5779worker1 # [ 129.479285] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:19Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05780worker1 # [ 129.486574] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:19Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15781worker1 # [ 129.525134] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577205782worker1 # [ 129.540451] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577205783client # copying 1 paths to 'lb:50051'...5784client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5785worker1 # [ 129.556107] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577205786worker1 # [ 129.571206] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5787worker1 # [ 129.596979] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:19Z level=info event=niks3_push_started pid=12305788worker1 # [ 129.746826] nix-grpc-daemon[1230]: time=2026-09-23T04:45:19.322Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5789worker1 # [ 129.851556] nix-daemon[835]: accepted connection from pid 1242, user nix-grpc-daemon (trusted)5790worker1 # [ 129.867900] nix-daemon[835]: reaped child process 1245, status = succeeded5791lb # [ 129.851939] niks3-server[1166]: 2026/09/23 04:45:19 INFO Received uploads request method=POST path=/api/pending_closures5792worker1 # [ 129.889101] nix-grpc-daemon[1230]: time=2026-09-23T04:45:19.464Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5793worker1 # [ 129.890586] nix-grpc-daemon[1230]: time=2026-09-23T04:45:19.464Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5794lb # [ 129.896776] niks3-server[1166]: 2026/09/23 04:45:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5795worker1 # [ 129.932059] nix-grpc-daemon[1230]: time=2026-09-23T04:45:19.507Z level=INFO msg="Uploading 1 narinfos"5796lb # [ 129.908164] niks3-server[1166]: 2026/09/23 04:45:19 INFO Signed narinfos id=43 count=15797lb # [ 129.919007] niks3-server[1166]: 2026/09/23 04:45:19 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5798lb # [ 129.932407] niks3-server[1166]: 2026/09/23 04:45:19 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5799lb # [ 129.945547] niks3-server[1166]: 2026/09/23 04:45:19 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5800lb # [ 129.956350] niks3-server[1166]: 2026/09/23 04:45:19 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5801lb # [ 129.961165] niks3-server[1166]: 2026/09/23 04:45:19 INFO Completed upload id=435802worker1 # [ 129.986178] nix-grpc-daemon[1230]: time=2026-09-23T04:45:19.561Z level=INFO msg="Upload complete. (235ms)"5803worker1 # [ 129.997831] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57720 duration_s=0 paths=1 nar_bytes_in=5205804worker1 # [ 130.018299] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577205805worker1 # [ 130.024850] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5806worker1 # [ 130.030202] nix-daemon[835]: reaped child process 1228, status = succeeded5807client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5808client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5809worker1 # [ 130.044820] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5810client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5811client # farm-a> LOG-farm-a-on-standby5812worker1 # [ 130.879379] nix-daemon[835]: reaped child process 1249, status = succeeded5813worker1 # [ 130.991554] nix-daemon[835]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5814worker1 # [ 131.012887] nix-daemon[835]: reaped child process 1268, status = succeeded5815lb # [ 130.987205] niks3-server[1166]: 2026/09/23 04:45:20 INFO Received uploads request method=POST path=/api/pending_closures5816worker1 # [ 131.023611] nix-grpc-daemon[1230]: time=2026-09-23T04:45:20.597Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5817worker1 # [ 131.026200] nix-grpc-daemon[1230]: time=2026-09-23T04:45:20.597Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5818lb # [ 131.019093] niks3-server[1166]: 2026/09/23 04:45:20 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5819worker1 # [ 131.069865] nix-grpc-daemon[1230]: time=2026-09-23T04:45:20.645Z level=INFO msg="Uploading 1 narinfos"5820lb # [ 131.045139] niks3-server[1166]: 2026/09/23 04:45:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5821lb # [ 131.054331] niks3-server[1166]: 2026/09/23 04:45:20 INFO Signed narinfos id=44 count=15822lb # [ 131.059632] niks3-server[1166]: 2026/09/23 04:45:20 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5823worker1 # [ 131.099261] nix-grpc-daemon[1230]: time=2026-09-23T04:45:20.675Z level=INFO msg="Upload complete. (243ms)"5824lb # [ 131.076371] niks3-server[1166]: 2026/09/23 04:45:20 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5825lb # [ 131.079652] niks3-server[1166]: 2026/09/23 04:45:20 INFO Completed upload id=445826lb # [ 131.092271] niks3-server[1166]: 2026/09/23 04:45:20 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5827worker1 # [ 131.120559] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57720 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15828lb # [ 131.106924] niks3-server[1166]: 2026/09/23 04:45:20 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5829worker1 # [ 131.132573] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:20Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05830worker1 # [ 131.138834] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:20Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15831worker1 # [ 131.143290] nix-daemon[835]: reaped child process 1247, status = succeeded5832worker1 # [ 131.172978] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577205833worker1 # [ 131.187911] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577205834worker1 # [ 131.204622] nix-grpc-daemon[1230]: time=2026-09-23T04:45:20.780Z level=INFO msg="All 1 paths already cached"5835client # copying 1 paths to 'lb:50051'...5836client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5837worker1 # [ 131.215881] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577205838worker1 # [ 131.229396] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5839worker1 # [ 131.378412] nix-daemon[835]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5840worker1 # [ 131.399007] nix-daemon[835]: reaped child process 1277, status = succeeded5841lb # [ 131.377730] niks3-server[1166]: 2026/09/23 04:45:20 INFO Received uploads request method=POST path=/api/pending_closures5842worker1 # [ 131.414818] nix-grpc-daemon[1230]: time=2026-09-23T04:45:20.990Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5843worker1 # [ 131.417448] nix-grpc-daemon[1230]: time=2026-09-23T04:45:20.992Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5844lb # [ 131.412866] niks3-server[1166]: 2026/09/23 04:45:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5845worker1 # [ 131.447660] nix-grpc-daemon[1230]: time=2026-09-23T04:45:21.023Z level=INFO msg="Uploading 1 narinfos"5846lb # [ 131.430179] niks3-server[1166]: 2026/09/23 04:45:20 INFO Signed narinfos id=45 count=15847lb # [ 131.436953] niks3-server[1166]: 2026/09/23 04:45:20 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5848lb # [ 131.443899] niks3-server[1166]: 2026/09/23 04:45:21 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5849lb # [ 131.456591] niks3-server[1166]: 2026/09/23 04:45:21 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5850worker1 # [ 131.481376] nix-grpc-daemon[1230]: time=2026-09-23T04:45:21.057Z level=INFO msg="Upload complete. (236ms)"5851lb # [ 131.461214] niks3-server[1166]: 2026/09/23 04:45:21 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5852lb # [ 131.465490] niks3-server[1166]: 2026/09/23 04:45:21 INFO Completed upload id=455853worker1 # [ 131.490253] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57720 duration_s=0 paths=1 nar_bytes_in=6405854worker1 # [ 131.508347] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577205855worker1 # [ 131.517533] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5856worker1 # [ 131.523073] nix-daemon[835]: reaped child process 1272, status = succeeded5857client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5858client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5859worker1 # [ 131.538988] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5860client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5861client # farm-b> LOG-farm-b-on-standby5862worker1 # [ 133.236713] nix-daemon[835]: reaped child process 1281, status = succeeded5863worker1 # [ 133.324104] nix-daemon[835]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5864worker1 # [ 133.341059] nix-daemon[835]: reaped child process 1300, status = succeeded5865lb # [ 133.323065] niks3-server[1166]: 2026/09/23 04:45:22 INFO Received uploads request method=POST path=/api/pending_closures5866worker1 # [ 133.358939] nix-grpc-daemon[1230]: time=2026-09-23T04:45:22.933Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5867worker1 # [ 133.360723] nix-grpc-daemon[1230]: time=2026-09-23T04:45:22.933Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5868lb # [ 133.364393] niks3-server[1166]: 2026/09/23 04:45:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5869worker1 # [ 133.399863] nix-grpc-daemon[1230]: time=2026-09-23T04:45:22.975Z level=INFO msg="Uploading 1 narinfos"5870lb # [ 133.382772] niks3-server[1166]: 2026/09/23 04:45:22 INFO Signed narinfos id=46 count=15871lb # [ 133.391698] niks3-server[1166]: 2026/09/23 04:45:22 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5872lb # [ 133.400480] niks3-server[1166]: 2026/09/23 04:45:22 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5873worker1 # [ 133.434478] nix-grpc-daemon[1230]: time=2026-09-23T04:45:23.010Z level=INFO msg="Upload complete. (284ms)"5874lb # [ 133.410930] niks3-server[1166]: 2026/09/23 04:45:22 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5875lb # [ 133.419967] niks3-server[1166]: 2026/09/23 04:45:22 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5876lb # [ 133.427196] niks3-server[1166]: 2026/09/23 04:45:22 INFO Completed upload id=465877lb # [ 133.429824] niks3-server[1166]: 2026/09/23 04:45:22 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5878worker1 # [ 133.450976] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57720 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15879worker1 # [ 133.472300] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:23Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05880worker1 # [ 133.478525] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:23Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15881worker1 # [ 133.483419] nix-daemon[835]: reaped child process 1279, status = succeeded5882worker1 # [ 133.513120] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577205883worker1 # [ 133.528804] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577205884worker1 # [ 133.543480] nix-grpc-daemon[1230]: time=2026-09-23T04:45:23.119Z level=INFO msg="All 2 paths already cached"5885client # copying 1 paths to 'lb:50051'...5886client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5887worker1 # [ 133.554994] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577205888worker1 # [ 133.569022] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5889worker1 # [ 133.721210] nix-daemon[835]: accepted connection from pid 1306, user nix-grpc-daemon (trusted)5890worker1 # [ 133.741117] nix-daemon[835]: reaped child process 1309, status = succeeded5891lb # [ 133.721063] niks3-server[1166]: 2026/09/23 04:45:23 INFO Received uploads request method=POST path=/api/pending_closures5892worker1 # [ 133.755245] nix-grpc-daemon[1230]: time=2026-09-23T04:45:23.330Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5893worker1 # [ 133.757436] nix-grpc-daemon[1230]: time=2026-09-23T04:45:23.330Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5894worker1 # [ 133.785606] nix-grpc-daemon[1230]: time=2026-09-23T04:45:23.361Z level=INFO msg="Uploading 1 narinfos"5895lb # [ 133.750990] niks3-server[1166]: 2026/09/23 04:45:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5896lb # [ 133.767905] niks3-server[1166]: 2026/09/23 04:45:23 INFO Signed narinfos id=47 count=15897lb # [ 133.775381] niks3-server[1166]: 2026/09/23 04:45:23 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5898lb # [ 133.782682] niks3-server[1166]: 2026/09/23 04:45:23 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5899lb # [ 133.792592] niks3-server[1166]: 2026/09/23 04:45:23 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5900worker1 # [ 133.817425] nix-grpc-daemon[1230]: time=2026-09-23T04:45:23.393Z level=INFO msg="Upload complete. (232ms)"5901lb # [ 133.800195] niks3-server[1166]: 2026/09/23 04:45:23 INFO Completed upload id=475902lb # [ 133.804446] niks3-server[1166]: 2026/09/23 04:45:23 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5903worker1 # [ 133.829516] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57720 duration_s=0 paths=1 nar_bytes_in=7685904worker1 # [ 133.843466] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577205905worker1 # [ 133.849833] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5906worker1 # [ 133.854593] nix-daemon[835]: reaped child process 1304, status = succeeded5907client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5908worker1 # [ 133.868168] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5909client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5910client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5911client # farm-top> LOG-farm-top-on-standby5912worker1 # [ 134.535671] nix-daemon[835]: reaped child process 1313, status = succeeded5913worker1 # [ 134.654625] nix-daemon[835]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5914worker1 # [ 134.675755] nix-daemon[835]: reaped child process 1332, status = succeeded5915lb # [ 134.651208] niks3-server[1166]: 2026/09/23 04:45:24 INFO Received uploads request method=POST path=/api/pending_closures5916worker1 # [ 134.687101] nix-grpc-daemon[1230]: time=2026-09-23T04:45:24.261Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5917worker1 # [ 134.688852] nix-grpc-daemon[1230]: time=2026-09-23T04:45:24.261Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5918lb # [ 134.692581] niks3-server[1166]: 2026/09/23 04:45:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5919worker1 # [ 134.727726] nix-grpc-daemon[1230]: time=2026-09-23T04:45:24.303Z level=INFO msg="Uploading 1 narinfos"5920lb # [ 134.706302] niks3-server[1166]: 2026/09/23 04:45:24 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5921lb # [ 134.716762] niks3-server[1166]: 2026/09/23 04:45:24 INFO Signed narinfos id=48 count=15922lb # [ 134.719000] niks3-server[1166]: 2026/09/23 04:45:24 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5923lb # [ 134.729578] niks3-server[1166]: 2026/09/23 04:45:24 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5924lb # [ 134.733054] niks3-server[1166]: 2026/09/23 04:45:24 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5925lb # [ 134.737662] niks3-server[1166]: 2026/09/23 04:45:24 INFO Completed upload id=485926worker1 # [ 134.761623] nix-grpc-daemon[1230]: time=2026-09-23T04:45:24.337Z level=INFO msg="Upload complete. (280ms)"5927lb # [ 134.739424] niks3-server[1166]: 2026/09/23 04:45:24 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5928worker1 # [ 134.777079] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57720 duration_s=0 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15929worker1 # [ 134.791165] nix-daemon[835]: reaped child process 1311, status = succeeded5930client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.02 seconds)5931client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5932client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.17 seconds)5933worker2 # [ 135.024977] systemd[1]: Listening on Nix worker-protocol over gRPC.5934worker2 # [ 135.033167] systemd[1]: Starting scheduler-token.service...5935worker2 # [ 135.112644] systemd[1]: scheduler-token.service: Deactivated successfully.5936worker2 # [ 135.120416] systemd[1]: Finished scheduler-token.service.5937worker2 # [ 135.122131] systemd[1]: scheduler-token.service: Consumed 45ms CPU time over 83ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5938worker2 # [ 135.129155] systemd[1]: Starting Nix worker-protocol over gRPC...5939worker2 # [ 135.252465] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5940worker2 # [ 135.254977] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5941worker2 # [ 135.258905] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15942worker2 # [ 135.264218] nix-grpc-daemon[1532]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5943worker2 # [ 135.265676] nix-grpc-daemon[1532]: E0000 00:00:1790138724.831505 1532 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5944lb: must succeed: curl -sf localhost:9901/clusters5945worker2 # [ 135.309656] nix-grpc-daemon[1532]: E0000 00:00:1790138724.838015 1532 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5946worker2 # [ 135.311962] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5947worker2 # [ 135.315120] nix-grpc-daemon[1532]: E0000 00:00:1790138724.845401 1572 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5948worker2 # [ 135.318217] nix-grpc-daemon[1532]: E0000 00:00:1790138724.862462 1572 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5949worker2 # [ 135.320757] nix-grpc-daemon[1532]: E0000 00:00:1790138724.862673 1572 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5950worker2 # [ 135.323136] systemd[1]: Started Nix worker-protocol over gRPC.5951worker2 # [ 135.323977] nix-grpc-daemon[1532]: E0000 00:00:1790138724.882370 1572 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5952worker2 # [ 135.331949] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:24Z level=info event=scheduler_connected addr=lb:500515953worker1 # [ 135.384133] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:24Z level=info event=oidc_jwks provider=mock keys=15954worker1 # [ 135.390307] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:523025955worker1 # [ 135.395684] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05956lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5957worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5958worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5959lb: must succeed: curl -sf localhost:9901/clusters5960lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5961lb: must succeed: curl -sf localhost:9901/clusters5962lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5963worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5964worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5965(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.56 seconds)5966subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5967client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5968client # [ 135.691071] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5969client # Running as unit: rs.service; invocation ID: c58d79605ca74fc993fc37729340fa155970client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5971client # [ 135.903642] nix[1403]: don't know how to build these paths:5972client # [ 135.904989] nix[1403]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5973worker1 # [ 135.936380] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577205974worker1 # [ 135.950269] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:25Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05975worker1 # [ 135.957389] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:25Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15976worker1 # [ 135.997259] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577205977worker1 # [ 136.012913] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577205978client # [ 135.992452] nix[1403]: copying 1 paths to 'lb:50051'...5979worker1 # [ 136.028548] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577205980client # [ 135.996717] nix[1403]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5981worker1 # [ 136.044473] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5982worker1 # [ 136.208149] nix-daemon[835]: accepted connection from pid 1345, user nix-grpc-daemon (trusted)5983worker1 # [ 136.228607] nix-daemon[835]: reaped child process 1348, status = succeeded5984lb # [ 136.203467] niks3-server[1166]: 2026/09/23 04:45:25 INFO Received uploads request method=POST path=/api/pending_closures5985worker1 # [ 136.238328] nix-grpc-daemon[1230]: time=2026-09-23T04:45:25.813Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5986worker1 # [ 136.240305] nix-grpc-daemon[1230]: time=2026-09-23T04:45:25.815Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5987lb # [ 136.237906] niks3-server[1166]: 2026/09/23 04:45:25 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5988worker1 # [ 136.272722] nix-grpc-daemon[1230]: time=2026-09-23T04:45:25.848Z level=INFO msg="Uploading 1 narinfos"5989lb # [ 136.250238] niks3-server[1166]: 2026/09/23 04:45:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5990lb # [ 136.259514] niks3-server[1166]: 2026/09/23 04:45:25 INFO Signed narinfos id=49 count=15991lb # [ 136.265683] niks3-server[1166]: 2026/09/23 04:45:25 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5992lb # [ 136.276345] niks3-server[1166]: 2026/09/23 04:45:25 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5993worker1 # [ 136.307031] nix-grpc-daemon[1230]: time=2026-09-23T04:45:25.882Z level=INFO msg="Upload complete. (246ms)"5994lb # [ 136.283361] niks3-server[1166]: 2026/09/23 04:45:25 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5995lb # [ 136.288774] niks3-server[1166]: 2026/09/23 04:45:25 INFO Completed upload id=495996worker1 # [ 136.316308] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57720 duration_s=0 paths=1 nar_bytes_in=4165997worker1 # [ 136.329042] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577205998worker1 # [ 136.344101] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5999worker1 # [ 136.347434] nix-daemon[835]: reaped child process 1343, status = succeeded6000worker1 # [ 136.360305] nix-daemon[835]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)6001client # [ 136.324410] nix[1403]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...6002lb: must succeed: curl -sf localhost:9901/clusters6003lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6004lb: must succeed: curl -sf localhost:9901/clusters6005lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6006lb: must succeed: curl -sf localhost:9901/clusters6007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6008lb: must succeed: curl -sf localhost:9901/clusters6009lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6010worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6011worker2 # [ 138.504776] systemd-journald[272]: Received client request to rotate journal, rotating.6012worker2 # [ 138.526235] systemd-journald[272]: Vacuuming done, freed 0B of archived journals from /var/log/journal/a2c9aeb4689b49769216a407ae57b6ad.6013worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)6014worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6015worker1 # [ 138.613404] systemd[1]: Stopping Nix worker-protocol over gRPC...6016worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)6017client: must succeed: sleep 5.06018lb # [ 138.661699] envoy[897]: conn peer=[2001:db8:1::1]:45010 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9576 tx=4399 ms=57686019worker1 # [ 138.897499] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:28Z level=info event=draining6020client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6021worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6022worker1 # [ 144.059444] nix-daemon[835]: reaped child process 1352, status = succeeded6023worker1 # [ 144.132106] nix-daemon[835]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6024worker1 # [ 144.151947] nix-daemon[835]: reaped child process 1396, status = succeeded6025lb # [ 144.126866] niks3-server[1166]: 2026/09/23 04:45:33 INFO Received uploads request method=POST path=/api/pending_closures6026worker1 # [ 144.163456] nix-grpc-daemon[1230]: time=2026-09-23T04:45:33.737Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6027worker1 # [ 144.166470] nix-grpc-daemon[1230]: time=2026-09-23T04:45:33.738Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"6028lb # [ 144.172388] niks3-server[1166]: 2026/09/23 04:45:33 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv6029worker1 # [ 144.207953] nix-grpc-daemon[1230]: time=2026-09-23T04:45:33.783Z level=INFO msg="Uploading 1 narinfos"6030lb # [ 144.186883] niks3-server[1166]: 2026/09/23 04:45:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6031lb # [ 144.195971] niks3-server[1166]: 2026/09/23 04:45:33 INFO Signed narinfos id=50 count=16032lb # [ 144.199133] niks3-server[1166]: 2026/09/23 04:45:33 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls6033lb # [ 144.209450] niks3-server[1166]: 2026/09/23 04:45:33 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6034lb # [ 144.214631] niks3-server[1166]: 2026/09/23 04:45:33 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6035lb # [ 144.218630] niks3-server[1166]: 2026/09/23 04:45:33 INFO Completed upload id=506036worker1 # [ 144.243582] nix-grpc-daemon[1230]: time=2026-09-23T04:45:33.819Z level=INFO msg="Upload complete. (266ms)"6037lb # [ 144.221450] niks3-server[1166]: 2026/09/23 04:45:33 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo6038worker1 # [ 144.261165] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57720 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=16039worker1 # [ 144.275654] nix-daemon[835]: reaped child process 1350, status = succeeded6040client # [ 144.254788] systemd[1]: rs.service: Deactivated successfully.6041client # [ 144.259885] systemd[1]: rs.service: Consumed 201ms CPU time over 8.564s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6042worker1 # [ 144.898101] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:34Z level=info event=signal_exit6043lb # [ 145.185349] niks3-server[1166]: 2026/09/23 04:45:34 INFO lead: released remote=192.168.1.3:558006044lb # [ 145.294721] niks3-server[1166]: 2026/09/23 04:45:34 INFO lead: acquired remote=192.168.1.4:403786045worker2 # [ 145.283480] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:34Z level=info event=scheduler_take_over6046worker2 # [ 146.167147] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:35Z level=info event=scheduler_restarting addr=lb:500516047worker1 # [ 146.213464] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:35Z level=info event=scheduler_restarting6048worker1 # [ 146.218746] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:35Z level=info event=scheduler_restarting addr=lb:500516049worker1 # [ 146.223776] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:523026050worker1 # [ 146.229470] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:35Z level=info event=worker_gone addr=192.168.1.4:500516051worker1 # [ 146.234072] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:523026052worker1 # [ 146.239007] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:35Z level=info event=worker_gone addr=192.168.1.3:500516053worker1 # [ 146.262245] nix-daemon[835]: reaped child process 1200, status = succeeded6054worker2 # [ 146.370618] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:35Z level=info event=scheduler_connected addr=lb:500516055worker2 # [ 146.374851] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6056worker1 # [ 146.432521] nix-grpc-daemon[1147]: ts=2026-09-23T04:45:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6057worker2 # [ 146.571740] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:36Z level=info event=scheduler_connected addr=lb:500516058worker2 # [ 146.588139] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:36Z level=info event=oidc_jwks provider=mock keys=16059worker2 # [ 146.593943] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:432306060worker2 # [ 146.599390] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06061worker1 # [ 147.050055] nix-grpc-daemon[1147]: E0000 00:00:1790138736.625729 1193 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6062worker1 # [ 147.095364] nix-grpc-daemon[1147]: E0000 00:00:1790138736.671059 1193 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6063worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.25 seconds)6064client: waiting for success: ! systemctl is-active rs6065client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)6066client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6067client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)6068client: must fail: journalctl -u rs | grep -q 'reconnecting'6069client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6070lb: must succeed: curl -sf localhost:9901/clusters6071lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6072worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6073worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6074worker1 # [ 147.516653] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6075worker1 # [ 147.523082] systemd[1]: Stopped Nix worker-protocol over gRPC.6076worker1 # [ 147.528806] systemd[1]: nix-grpc-daemon.service: Consumed 1.597s CPU time over 32.800s wall clock time, 39.2M memory peak, 47.8K incoming IP traffic, 90.1K outgoing IP traffic.6077worker1 # [ 147.548490] systemd[1]: Starting Nix worker-protocol over gRPC...6078worker1 # [ 147.685910] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6079worker1 # [ 147.688211] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6080worker1 # [ 147.691934] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16081worker1 # [ 147.697458] nix-grpc-daemon[1403]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6082worker1 # [ 147.698895] nix-grpc-daemon[1403]: E0000 00:00:1790138737.272526 1403 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6083worker1 # [ 147.738664] nix-grpc-daemon[1403]: E0000 00:00:1790138737.278462 1403 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6084worker1 # [ 147.741874] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6085worker1 # [ 147.744089] nix-grpc-daemon[1403]: E0000 00:00:1790138737.284281 1444 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6086worker1 # [ 147.746600] nix-grpc-daemon[1403]: E0000 00:00:1790138737.299909 1444 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6087worker1 # [ 147.748872] nix-grpc-daemon[1403]: E0000 00:00:1790138737.300104 1444 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6088worker1 # [ 147.754193] nix-grpc-daemon[1403]: E0000 00:00:1790138737.314407 1444 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6089worker1 # [ 147.758473] systemd[1]: Started Nix worker-protocol over gRPC.6090worker1 # [ 147.759380] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:37Z level=info event=scheduler_connected addr=lb:500516091worker2 # [ 147.718692] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:432306092worker2 # [ 147.724595] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06093lb: must succeed: curl -sf localhost:9901/clusters6094lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6095worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6096worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6097(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.92 seconds)6098subtest: scheduler restart mid-build: build finishes, no double build6099client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid6100client # [ 148.611284] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.6101client # Running as unit: mid.service; invocation ID: f06d170b5b2a4de187a278729b420e726102client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)6103lb # [ 148.665032] envoy[897]: conn peer=[2001:db8:1::1]:47488 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1515 ms=83346104lb # [ 148.670197] envoy[897]: conn peer=[2001:db8:1::3]:59776 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2625 tx=938 ms=314816105worker1 # [ 148.851621] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:342186106worker1 # [ 148.863965] nix-daemon[835]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6107client # [ 148.847122] nix[1439]: don't know how to build these paths:6108client # [ 148.848667] nix[1439]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6109worker2 # [ 148.857040] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:38Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06110worker2 # [ 148.862463] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:38Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16111worker2 # [ 148.914226] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432466112worker2 # [ 148.928930] nix-daemon[859]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)6113worker2 # [ 148.951464] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)6114worker2 # [ 148.970129] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 2/5)6115worker2 # [ 148.979419] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 149 ms (attempt 3/5)6116worker2 # [ 149.130286] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 270 ms (attempt 4/5)6117worker2 # [ 149.402725] nix-grpc-daemon[1532]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6118worker2 # [ 149.409722] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6119worker2 # [ 149.427276] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432466120client # [ 149.446994] nix[1439]: copying 1 paths to 'lb:50051'...6121client # [ 149.452277] nix[1439]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...6122worker2 # [ 149.446246] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432466123worker2 # [ 149.460760] nix-daemon[859]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)6124worker2 # [ 149.500219] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:39Z level=info event=niks3_push_started pid=16616125worker2 # [ 149.660486] nix-grpc-daemon[1661]: time=2026-09-23T04:45:39.227Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6126worker2 # [ 149.768093] nix-daemon[859]: accepted connection from pid 1674, user nix-grpc-daemon (trusted)6127worker2 # [ 149.785881] nix-daemon[859]: reaped child process 1677, status = succeeded6128lb # [ 149.811546] niks3-server[1166]: 2026/09/23 04:45:39 INFO Received uploads request method=POST path=/api/pending_closures6129worker2 # [ 149.805058] nix-grpc-daemon[1661]: time=2026-09-23T04:45:39.372Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6130worker2 # [ 149.806572] nix-grpc-daemon[1661]: time=2026-09-23T04:45:39.372Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"6131lb # [ 149.851300] niks3-server[1166]: 2026/09/23 04:45:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6132worker2 # [ 149.847873] nix-grpc-daemon[1661]: time=2026-09-23T04:45:39.414Z level=INFO msg="Uploading 1 narinfos"6133lb # [ 149.865808] niks3-server[1166]: 2026/09/23 04:45:39 INFO Signed narinfos id=51 count=16134lb # [ 149.872360] niks3-server[1166]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls6135lb # [ 149.882574] niks3-server[1166]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst6136lb # [ 149.899103] niks3-server[1166]: 2026/09/23 04:45:39 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6137worker2 # [ 149.889519] nix-grpc-daemon[1661]: time=2026-09-23T04:45:39.457Z level=INFO msg="Upload complete. (225ms)"6138lb # [ 149.908059] niks3-server[1166]: 2026/09/23 04:45:39 INFO Completed upload id=516139worker2 # [ 149.895935] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43246 duration_s=0 paths=1 nar_bytes_in=4246140lb # [ 149.912562] niks3-server[1166]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo6141worker2 # [ 149.900907] nix-grpc-daemon[1532]: ts=2026-09-23T04:45:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432466142worker2 # [ 149.909148] nix-daemon[859]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)6143worker2 # [ 149.912799] nix-daemon[859]: reaped child process 1659, status = succeeded6144worker2 # [ 149.921899] nix-daemon[859]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)6145client # [ 149.927953] nix[1439]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6146lb: must succeed: curl -sf localhost:9901/clusters6147lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6148lb: must succeed: curl -sf localhost:9901/clusters6149lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6150worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6151worker2 # [ 151.284399] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1532 (nix-grpc-daemon) on client request.6152worker2 # [ 151.286009] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6153lb # [ 151.310677] niks3-server[1166]: 2026/09/23 04:45:40 INFO lead: released remote=192.168.1.4:403786154worker2 # [ 151.306059] systemd[1]: Starting scheduler-token.service...6155client # [ 151.310212] nix[1439]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6156worker1 # [ 151.344842] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6157client # [ 151.316669] nix[1439]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6158worker2 # [ 151.318239] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6159worker2 # [ 151.319728] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6160worker2 # [ 151.326427] systemd[1]: nix-grpc-daemon.service: Consumed 519ms CPU time over 16.179s wall clock time, 34.6M memory peak, 15.3K incoming IP traffic, 25.7K outgoing IP traffic.6161worker2 # [ 151.360791] nix-daemon[859]: reaped child process 1653, status = succeeded6162worker2 # [ 151.362015] nix-daemon[859]: reaped child process 1683, status = succeeded6163worker2 # [ 151.368433] nix-daemon[1685]: unexpected Nix daemon error: error: interrupted by the user6164worker2 # [ 151.380927] nix-daemon[859]: reaped child process 1685, status = failed with exit code 16165worker2 # [ 151.396961] systemd[1]: scheduler-token.service: Deactivated successfully.6166worker2 # [ 151.399124] systemd[1]: Finished scheduler-token.service.6167worker2 # [ 151.399914] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 91ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6168worker2 # [ 151.406034] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6169worker2 # [ 151.409762] systemd[1]: Starting scheduler-token.service...6170worker2 # [ 151.449429] systemd[1]: scheduler-token.service: Deactivated successfully.6171worker2 # [ 151.452074] systemd[1]: Finished scheduler-token.service.6172worker2 # [ 151.452900] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 45ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6173worker2 # [ 151.457369] systemd[1]: Starting Nix worker-protocol over gRPC...6174worker1 # [ 151.544851] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:41Z level=info event=scheduler_connected addr=lb:500516175worker2 # [ 151.560703] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6176worker2 # [ 151.562607] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6177worker2 # [ 151.566450] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16178worker2 # [ 151.572064] nix-grpc-daemon[1724]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6179worker2 # [ 151.573505] nix-grpc-daemon[1724]: E0000 00:00:1790138741.138956 1724 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6180lb # [ 151.609870] niks3-server[1166]: 2026/09/23 04:45:41 INFO lead: acquired remote=192.168.1.4:446526181worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.37 seconds)6182client: must succeed: sleep 5.06183worker2 # [ 151.620091] nix-grpc-daemon[1724]: E0000 00:00:1790138741.145141 1724 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6184worker2 # [ 151.622413] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6185worker2 # [ 151.630648] nix-grpc-daemon[1724]: E0000 00:00:1790138741.155978 1765 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6186worker2 # [ 151.633718] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=scheduler_take_over6187worker2 # [ 151.635002] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:418546188worker2 # [ 151.636586] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06189worker2 # [ 151.641168] nix-grpc-daemon[1724]: E0000 00:00:1790138741.179418 1765 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6190worker2 # [ 151.645851] nix-grpc-daemon[1724]: E0000 00:00:1790138741.179552 1765 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6191worker2 # [ 151.648773] systemd[1]: Started Nix worker-protocol over gRPC.6192worker2 # [ 151.650105] nix-grpc-daemon[1724]: E0000 00:00:1790138741.198113 1765 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6193worker2 # [ 151.652839] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=scheduler_connected addr=lb:500516194worker2 # [ 151.655898] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=oidc_jwks provider=mock keys=16195worker2 # [ 151.657335] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:418546196worker2 # [ 151.658999] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06197worker2 # [ 151.814145] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06198worker2 # [ 151.820245] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:41Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16199worker1 # [ 151.901678] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342186200worker1 # [ 151.918297] nix-grpc-daemon[1403]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)6201worker1 # [ 151.957647] nix-grpc-daemon[1403]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 199 ms (attempt 2/5)6202worker1 # [ 152.158857] nix-grpc-daemon[1403]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 121 ms (attempt 3/5)6203worker1 # [ 152.289463] nix-grpc-daemon[1403]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 4/5)6204worker1 # [ 152.302028] nix-grpc-daemon[1403]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6205worker1 # [ 152.306809] nix-grpc-daemon[1403]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6206worker1 # [ 152.320247] nix-grpc-daemon[1403]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...6207worker1 # [ 152.349397] nix-daemon[835]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6208client # [ 152.332899] nix[1439]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6209worker1 # [ 152.371120] nix-daemon[835]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6210client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6211worker1 # [ 156.998089] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:46Z level=info event=niks3_push_started pid=15066212client: waiting for success: ! systemctl is-active mid6213worker1 # [ 157.027343] nix-daemon[835]: reaped child process 1481, status = succeeded6214worker1 # [ 157.169540] nix-grpc-daemon[1506]: time=2026-09-23T04:45:46.744Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6215worker1 # [ 157.303050] nix-daemon[835]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6216worker1 # [ 157.323089] nix-daemon[835]: reaped child process 1522, status = succeeded6217lb # [ 157.305147] niks3-server[1166]: 2026/09/23 04:45:46 INFO Received uploads request method=POST path=/api/pending_closures6218worker1 # [ 157.343921] nix-grpc-daemon[1506]: time=2026-09-23T04:45:46.917Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6219worker1 # [ 157.348119] nix-grpc-daemon[1506]: time=2026-09-23T04:45:46.917Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6220worker1 # [ 157.394627] nix-grpc-daemon[1506]: time=2026-09-23T04:45:46.970Z level=INFO msg="Uploading 1 narinfos"6221lb # [ 157.360885] niks3-server[1166]: 2026/09/23 04:45:46 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6222lb # [ 157.372928] niks3-server[1166]: 2026/09/23 04:45:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6223lb # [ 157.384101] niks3-server[1166]: 2026/09/23 04:45:46 INFO Signed narinfos id=52 count=16224lb # [ 157.389223] niks3-server[1166]: 2026/09/23 04:45:46 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6225lb # [ 157.402909] niks3-server[1166]: 2026/09/23 04:45:46 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6226lb # [ 157.419732] niks3-server[1166]: 2026/09/23 04:45:46 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6227lb # [ 157.426362] niks3-server[1166]: 2026/09/23 04:45:46 INFO Completed upload id=526228worker1 # [ 157.451458] nix-grpc-daemon[1506]: time=2026-09-23T04:45:47.027Z level=INFO msg="Upload complete. (277ms)"6229worker1 # [ 157.464383] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34218 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16230worker1 # [ 157.484423] nix-daemon[835]: reaped child process 1479, status = succeeded6231client # [ 157.472361] systemd[1]: mid.service: Deactivated successfully.6232client # [ 157.477832] systemd[1]: mid.service: Consumed 204ms CPU time over 8.865s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.2K outgoing IP traffic.6233client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)6234client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6235client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)6236(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.63 seconds)6237subtest: interrupting the client stops the build on the worker6238client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6239client # [ 158.216117] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6240client # Running as unit: intr.service; invocation ID: b2864c1ebefe46c3b05f1c0f0f3388456241client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6242worker2 # [ 158.389587] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418746243worker2 # [ 158.396779] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6244client # [ 158.420514] nix[1475]: don't know how to build these paths:6245client # [ 158.421607] nix[1475]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6246worker2 # [ 158.424621] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:47Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06247worker2 # [ 158.430445] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:47Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16248worker2 # [ 158.471824] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746249worker2 # [ 158.487391] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 1/5)6250worker2 # [ 158.552191] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 2/5)6251worker2 # [ 158.626833] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 3/5)6252lb # [ 158.663116] envoy[897]: conn peer=[2001:db8:1::4]:46000 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2722 tx=393 ms=159826253lb # [ 158.668327] envoy[897]: conn peer=[2001:db8:1::1]:52122 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3897 tx=1925 ms=86356254worker2 # [ 158.724914] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 4/5)6255worker2 # [ 158.750996] nix-grpc-daemon[1724]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6256worker2 # [ 158.757502] nix-grpc-daemon[1724]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6257worker2 # [ 158.775790] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418746258client # [ 158.796483] nix[1475]: copying 1 paths to 'lb:50051'...6259worker2 # [ 158.791062] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418746260client # [ 158.801459] nix[1475]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6261worker2 # [ 158.805534] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6262worker2 # [ 158.833039] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:48Z level=info event=niks3_push_started pid=18016263worker2 # [ 158.983270] nix-grpc-daemon[1801]: time=2026-09-23T04:45:48.550Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6264worker2 # [ 159.090319] nix-daemon[859]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)6265worker2 # [ 159.107563] nix-daemon[859]: reaped child process 1816, status = succeeded6266lb # [ 159.132694] niks3-server[1166]: 2026/09/23 04:45:48 INFO Received uploads request method=POST path=/api/pending_closures6267worker2 # [ 159.128315] nix-grpc-daemon[1801]: time=2026-09-23T04:45:48.696Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6268worker2 # [ 159.133079] nix-grpc-daemon[1801]: time=2026-09-23T04:45:48.698Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6269lb # [ 159.175622] niks3-server[1166]: 2026/09/23 04:45:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6270worker2 # [ 159.172961] nix-grpc-daemon[1801]: time=2026-09-23T04:45:48.740Z level=INFO msg="Uploading 1 narinfos"6271lb # [ 159.192402] niks3-server[1166]: 2026/09/23 04:45:48 INFO Signed narinfos id=53 count=16272lb # [ 159.195209] niks3-server[1166]: 2026/09/23 04:45:48 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6273lb # [ 159.214028] niks3-server[1166]: 2026/09/23 04:45:48 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6274worker2 # [ 159.203022] nix-grpc-daemon[1801]: time=2026-09-23T04:45:48.771Z level=INFO msg="Upload complete. (216ms)"6275lb # [ 159.220597] niks3-server[1166]: 2026/09/23 04:45:48 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6276lb # [ 159.223921] niks3-server[1166]: 2026/09/23 04:45:48 INFO Completed upload id=536277lb # [ 159.227087] niks3-server[1166]: 2026/09/23 04:45:48 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6278worker2 # [ 159.212247] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=4246279worker2 # [ 159.226794] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746280worker2 # [ 159.233410] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6281worker2 # [ 159.238476] nix-daemon[859]: reaped child process 1799, status = succeeded6282worker2 # [ 159.248731] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6283client # [ 159.254656] nix[1475]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6284client: must succeed: systemctl kill -s INT intr6285worker2 # [ 160.703254] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:50Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6286client # [ 160.709565] systemd[1]: intr.service: Sent signal SIGINT to main process 1475 (nix) on client request.6287client # [ 160.718118] nix[1475]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6288client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)6289client # [ 160.732318] nix[1475]: error: interrupted by the user6290client # [ 160.755990] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6291client # [ 160.760876] systemd[1]: intr.service: Failed with result 'exit-code'.6292client # [ 160.763065] systemd[1]: intr.service: Consumed 171ms CPU time over 2.543s wall clock time, 41.5M memory peak, 6.4K incoming IP traffic, 8.4K outgoing IP traffic.6293worker2 # [ 160.875743] nix-daemon[1820]: unexpected Nix daemon error: error: interrupted by the user6294worker2 # [ 160.881554] nix-daemon[859]: reaped child process 1818, status = succeeded6295worker2 # [ 160.885936] nix-daemon[859]: reaped child process 1820, status = failed with exit code 16296(finished: subtest: interrupting the client stops the build on the worker, in 3.82 seconds)6297subtest: a deploy mid-build drains: the build finishes, the new generation takes over6298client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6299client # [ 162.063326] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6300client # Running as unit: sw.service; invocation ID: 4ae71c5dbd834bbabf69e5f6984c29126301client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)6302client # [ 162.255397] nix[1496]: don't know how to build these paths:6303client # [ 162.256406] nix[1496]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6304worker1 # [ 162.289843] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:342186305worker2 # [ 162.259699] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:51Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06306worker2 # [ 162.263721] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:51Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16307worker1 # [ 162.344909] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342186308worker1 # [ 162.361734] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342186309client # [ 162.339348] nix[1496]: copying 1 paths to 'lb:50051'...6310client # [ 162.344389] nix[1496]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6311worker1 # [ 162.379228] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342186312worker1 # [ 162.394125] nix-daemon[835]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6313worker1 # [ 162.558963] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6314worker1 # [ 162.577649] nix-daemon[835]: reaped child process 1553, status = succeeded6315lb # [ 162.559187] niks3-server[1166]: 2026/09/23 04:45:52 INFO Received uploads request method=POST path=/api/pending_closures6316worker1 # [ 162.594401] nix-grpc-daemon[1506]: time=2026-09-23T04:45:52.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6317worker1 # [ 162.596244] nix-grpc-daemon[1506]: time=2026-09-23T04:45:52.169Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6318lb # [ 162.593219] niks3-server[1166]: 2026/09/23 04:45:52 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6319worker1 # [ 162.631700] nix-grpc-daemon[1506]: time=2026-09-23T04:45:52.206Z level=INFO msg="Uploading 1 narinfos"6320lb # [ 162.608774] niks3-server[1166]: 2026/09/23 04:45:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6321lb # [ 162.617645] niks3-server[1166]: 2026/09/23 04:45:52 INFO Signed narinfos id=54 count=16322lb # [ 162.623852] niks3-server[1166]: 2026/09/23 04:45:52 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6323lb # [ 162.635461] niks3-server[1166]: 2026/09/23 04:45:52 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6324lb # [ 162.641121] niks3-server[1166]: 2026/09/23 04:45:52 INFO Completed upload id=546325lb # [ 162.644574] niks3-server[1166]: 2026/09/23 04:45:52 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6326worker1 # [ 162.665122] nix-grpc-daemon[1506]: time=2026-09-23T04:45:52.240Z level=INFO msg="Upload complete. (251ms)"6327worker1 # [ 162.677174] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34218 duration_s=0 paths=1 nar_bytes_in=4166328worker1 # [ 162.690745] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342186329worker1 # [ 162.697521] nix-daemon[835]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6330worker1 # [ 162.702410] nix-daemon[835]: reaped child process 1548, status = succeeded6331worker1 # [ 162.713105] nix-daemon[835]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6332client # [ 162.677367] nix[1496]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6333worker1: must succeed: systemctl reload nix-grpc-daemon.service6334worker1 # [ 164.714684] systemd[1]: Reloading Nix worker-protocol over gRPC...6335worker1 # [ 164.751320] systemd[1]: Reloaded Nix worker-protocol over gRPC.6336worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds)6337worker1: must succeed: pgrep -f 'read -t [9]0 x'6338worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6339client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26340client # don't know how to build these paths:6341client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6342worker2 # [ 164.989227] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418746343worker1 # [ 165.037821] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342186344worker2 # [ 165.016138] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:54Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06345worker2 # [ 165.023469] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:54Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16346worker2 # [ 165.067282] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746347worker2 # [ 165.088636] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418746348client # copying 1 paths to 'lb:50051'...6349worker2 # [ 165.104540] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418746350client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6351worker2 # [ 165.128168] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6352worker2 # [ 165.300441] nix-daemon[859]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6353worker2 # [ 165.318918] nix-daemon[859]: reaped child process 1879, status = succeeded6354lb # [ 165.340717] niks3-server[1166]: 2026/09/23 04:45:54 INFO Received uploads request method=POST path=/api/pending_closures6355worker2 # [ 165.334141] nix-grpc-daemon[1801]: time=2026-09-23T04:45:54.901Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6356worker2 # [ 165.335639] nix-grpc-daemon[1801]: time=2026-09-23T04:45:54.901Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6357lb # [ 165.371768] niks3-server[1166]: 2026/09/23 04:45:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6358worker2 # [ 165.364777] nix-grpc-daemon[1801]: time=2026-09-23T04:45:54.932Z level=INFO msg="Uploading 1 narinfos"6359lb # [ 165.387306] niks3-server[1166]: 2026/09/23 04:45:54 INFO Signed narinfos id=55 count=16360lb # [ 165.396923] niks3-server[1166]: 2026/09/23 04:45:54 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6361lb # [ 165.406591] niks3-server[1166]: 2026/09/23 04:45:54 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6362lb # [ 165.423057] niks3-server[1166]: 2026/09/23 04:45:54 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6363lb # [ 165.436817] niks3-server[1166]: 2026/09/23 04:45:54 INFO Completed upload id=556364worker2 # [ 165.414968] nix-grpc-daemon[1801]: time=2026-09-23T04:45:54.981Z level=INFO msg="Upload complete. (267ms)"6365lb # [ 165.443561] niks3-server[1166]: 2026/09/23 04:45:55 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6366worker2 # [ 165.425083] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=5286367worker2 # [ 165.443113] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746368worker2 # [ 165.450345] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6369worker2 # [ 165.455901] nix-daemon[859]: reaped child process 1873, status = succeeded6370client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6371client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6372worker2 # [ 165.470027] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6373worker1 # [ 165.713949] nix-grpc-daemon[1403]: ts=2026-09-23T04:45:55Z level=info event=draining6374client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6375client # farm-a> LOG-farm-a-during-drain6376worker2 # [ 166.944297] nix-daemon[859]: reaped child process 1883, status = succeeded6377worker2 # [ 167.049909] nix-daemon[859]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6378worker2 # [ 167.070435] nix-daemon[859]: reaped child process 1902, status = succeeded6379lb # [ 167.093013] niks3-server[1166]: 2026/09/23 04:45:56 INFO Received uploads request method=POST path=/api/pending_closures6380worker2 # [ 167.087640] nix-grpc-daemon[1801]: time=2026-09-23T04:45:56.653Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6381worker2 # [ 167.089878] nix-grpc-daemon[1801]: time=2026-09-23T04:45:56.653Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6382lb # [ 167.124688] niks3-server[1166]: 2026/09/23 04:45:56 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6383worker2 # [ 167.129166] nix-grpc-daemon[1801]: time=2026-09-23T04:45:56.697Z level=INFO msg="Uploading 1 narinfos"6384lb # [ 167.145742] niks3-server[1166]: 2026/09/23 04:45:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6385lb # [ 167.157452] niks3-server[1166]: 2026/09/23 04:45:56 INFO Signed narinfos id=56 count=16386lb # [ 167.164348] niks3-server[1166]: 2026/09/23 04:45:56 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6387lb # [ 167.174439] niks3-server[1166]: 2026/09/23 04:45:56 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6388lb # [ 167.187092] niks3-server[1166]: 2026/09/23 04:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6389lb # [ 167.193912] niks3-server[1166]: 2026/09/23 04:45:56 INFO Completed upload id=566390worker2 # [ 167.177315] nix-grpc-daemon[1801]: time=2026-09-23T04:45:56.745Z level=INFO msg="Upload complete. (293ms)"6391lb # [ 167.197687] niks3-server[1166]: 2026/09/23 04:45:56 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6392worker2 # [ 167.195520] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16393worker2 # [ 167.214949] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:56Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06394worker2 # [ 167.219182] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:56Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16395worker2 # [ 167.223542] nix-daemon[859]: reaped child process 1881, status = succeeded6396worker2 # [ 167.239302] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746397worker2 # [ 167.251532] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418746398worker2 # [ 167.263198] nix-grpc-daemon[1801]: time=2026-09-23T04:45:56.831Z level=INFO msg="All 1 paths already cached"6399client # copying 1 paths to 'lb:50051'...6400worker2 # [ 167.271954] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418746401client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6402worker2 # [ 167.296196] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6403worker2 # [ 167.451482] nix-daemon[859]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6404worker2 # [ 167.470994] nix-daemon[859]: reaped child process 1911, status = succeeded6405lb # [ 167.491808] niks3-server[1166]: 2026/09/23 04:45:57 INFO Received uploads request method=POST path=/api/pending_closures6406worker2 # [ 167.484070] nix-grpc-daemon[1801]: time=2026-09-23T04:45:57.051Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6407worker2 # [ 167.485598] nix-grpc-daemon[1801]: time=2026-09-23T04:45:57.051Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6408worker2 # [ 167.515986] nix-grpc-daemon[1801]: time=2026-09-23T04:45:57.083Z level=INFO msg="Uploading 1 narinfos"6409lb # [ 167.524813] niks3-server[1166]: 2026/09/23 04:45:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6410lb # [ 167.540177] niks3-server[1166]: 2026/09/23 04:45:57 INFO Signed narinfos id=57 count=16411lb # [ 167.543219] niks3-server[1166]: 2026/09/23 04:45:57 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6412lb # [ 167.562183] niks3-server[1166]: 2026/09/23 04:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6413worker2 # [ 167.550568] nix-grpc-daemon[1801]: time=2026-09-23T04:45:57.118Z level=INFO msg="Upload complete. (238ms)"6414lb # [ 167.569024] niks3-server[1166]: 2026/09/23 04:45:57 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6415lb # [ 167.573734] niks3-server[1166]: 2026/09/23 04:45:57 INFO Completed upload id=576416lb # [ 167.576983] niks3-server[1166]: 2026/09/23 04:45:57 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6417worker2 # [ 167.560452] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=6406418worker2 # [ 167.575701] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746419worker2 # [ 167.582502] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6420worker2 # [ 167.588704] nix-daemon[859]: reaped child process 1906, status = succeeded6421client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6422client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6423worker2 # [ 167.601475] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6424client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6425client # farm-b> LOG-farm-b-during-drain6426lb # [ 168.664890] envoy[897]: conn peer=[2001:db8:1::1]:59950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=828 ms=23476427worker2 # [ 168.689420] nix-daemon[859]: reaped child process 1915, status = succeeded6428worker2 # [ 168.755288] nix-daemon[859]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6429worker2 # [ 168.773246] nix-daemon[859]: reaped child process 1934, status = succeeded6430lb # [ 168.796495] niks3-server[1166]: 2026/09/23 04:45:58 INFO Received uploads request method=POST path=/api/pending_closures6431worker2 # [ 168.791017] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.358Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6432worker2 # [ 168.792691] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.358Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6433lb # [ 168.838447] niks3-server[1166]: 2026/09/23 04:45:58 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6434worker2 # [ 168.832101] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.400Z level=INFO msg="Uploading 1 narinfos"6435lb # [ 168.853876] niks3-server[1166]: 2026/09/23 04:45:58 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6436lb # [ 168.861837] niks3-server[1166]: 2026/09/23 04:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6437lb # [ 168.872596] niks3-server[1166]: 2026/09/23 04:45:58 INFO Signed narinfos id=58 count=16438lb # [ 168.874491] niks3-server[1166]: 2026/09/23 04:45:58 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6439worker2 # [ 168.862305] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.430Z level=INFO msg="Upload complete. (250ms)"6440lb # [ 168.879583] niks3-server[1166]: 2026/09/23 04:45:58 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6441lb # [ 168.882643] niks3-server[1166]: 2026/09/23 04:45:58 INFO Completed upload id=586442lb # [ 168.886076] niks3-server[1166]: 2026/09/23 04:45:58 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6443worker2 # [ 168.875945] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16444worker2 # [ 168.892287] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:58Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06445worker2 # [ 168.895789] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:58Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16446worker2 # [ 168.899909] nix-daemon[859]: reaped child process 1913, status = succeeded6447worker2 # [ 168.930949] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746448worker2 # [ 168.939949] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418746449worker2 # [ 168.948860] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.516Z level=INFO msg="All 2 paths already cached"6450client # copying 1 paths to 'lb:50051'...6451client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6452worker2 # [ 168.957427] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418746453worker2 # [ 168.972692] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6454worker2 # [ 169.111117] nix-daemon[859]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6455worker2 # [ 169.127121] nix-daemon[859]: reaped child process 1943, status = succeeded6456lb # [ 169.151533] niks3-server[1166]: 2026/09/23 04:45:58 INFO Received uploads request method=POST path=/api/pending_closures6457worker2 # [ 169.143040] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.710Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6458worker2 # [ 169.144643] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.710Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6459lb # [ 169.179491] niks3-server[1166]: 2026/09/23 04:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6460worker2 # [ 169.169706] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.737Z level=INFO msg="Uploading 1 narinfos"6461lb # [ 169.193418] niks3-server[1166]: 2026/09/23 04:45:58 INFO Signed narinfos id=59 count=16462lb # [ 169.202544] niks3-server[1166]: 2026/09/23 04:45:58 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6463lb # [ 169.216394] niks3-server[1166]: 2026/09/23 04:45:58 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6464worker2 # [ 169.206000] nix-grpc-daemon[1801]: time=2026-09-23T04:45:58.774Z level=INFO msg="Upload complete. (222ms)"6465lb # [ 169.225726] niks3-server[1166]: 2026/09/23 04:45:58 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6466lb # [ 169.231245] niks3-server[1166]: 2026/09/23 04:45:58 INFO Completed upload id=596467worker2 # [ 169.216381] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=7686468lb # [ 169.235063] niks3-server[1166]: 2026/09/23 04:45:58 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6469worker2 # [ 169.227155] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746470worker2 # [ 169.237052] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6471worker2 # [ 169.240175] nix-daemon[859]: reaped child process 1938, status = succeeded6472client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6473worker2 # [ 169.251874] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6474client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6475client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6476client # farm-top> LOG-farm-top-during-drain6477worker2 # [ 170.096371] nix-daemon[859]: reaped child process 1947, status = succeeded6478worker2 # [ 170.185924] nix-daemon[859]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6479worker2 # [ 170.203023] nix-daemon[859]: reaped child process 1966, status = succeeded6480lb # [ 170.227102] niks3-server[1166]: 2026/09/23 04:45:59 INFO Received uploads request method=POST path=/api/pending_closures6481worker2 # [ 170.220390] nix-grpc-daemon[1801]: time=2026-09-23T04:45:59.787Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6482worker2 # [ 170.221890] nix-grpc-daemon[1801]: time=2026-09-23T04:45:59.787Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6483lb # [ 170.267692] niks3-server[1166]: 2026/09/23 04:45:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6484worker2 # [ 170.259957] nix-grpc-daemon[1801]: time=2026-09-23T04:45:59.827Z level=INFO msg="Uploading 1 narinfos"6485lb # [ 170.283388] niks3-server[1166]: 2026/09/23 04:45:59 INFO Signed narinfos id=60 count=16486lb # [ 170.293644] niks3-server[1166]: 2026/09/23 04:45:59 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6487lb # [ 170.303942] niks3-server[1166]: 2026/09/23 04:45:59 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6488lb # [ 170.313369] niks3-server[1166]: 2026/09/23 04:45:59 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6489worker2 # [ 170.301385] nix-grpc-daemon[1801]: time=2026-09-23T04:45:59.869Z level=INFO msg="Upload complete. (237ms)"6490lb # [ 170.320453] niks3-server[1166]: 2026/09/23 04:45:59 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6491lb # [ 170.327646] niks3-server[1166]: 2026/09/23 04:45:59 INFO Completed upload id=606492lb # [ 170.331392] niks3-server[1166]: 2026/09/23 04:45:59 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6493worker2 # [ 170.317044] nix-grpc-daemon[1724]: ts=2026-09-23T04:45:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16494worker2 # [ 170.331158] nix-daemon[859]: reaped child process 1945, status = succeeded6495client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.60 seconds)6496client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6497client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6498worker1: must succeed: pkill -f 'read -t [9]0 x'6499worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6500client: waiting for success: ! systemctl is-active sw6501worker1 # [ 170.784657] nix-daemon[835]: reaped child process 1557, status = succeeded6502worker1 # [ 170.884498] nix-daemon[835]: accepted connection from pid 1599, user nix-grpc-daemon (trusted)6503worker1 # [ 170.905950] nix-daemon[835]: reaped child process 1602, status = succeeded6504lb # [ 170.884702] niks3-server[1166]: 2026/09/23 04:46:00 INFO Received uploads request method=POST path=/api/pending_closures6505worker1 # [ 170.919281] nix-grpc-daemon[1506]: time=2026-09-23T04:46:00.494Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6506worker1 # [ 170.921507] nix-grpc-daemon[1506]: time=2026-09-23T04:46:00.494Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6507lb # [ 170.925554] niks3-server[1166]: 2026/09/23 04:46:00 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6508worker1 # [ 170.965683] nix-grpc-daemon[1506]: time=2026-09-23T04:46:00.541Z level=INFO msg="Uploading 1 narinfos"6509lb # [ 170.942095] niks3-server[1166]: 2026/09/23 04:46:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6510lb # [ 170.950279] niks3-server[1166]: 2026/09/23 04:46:00 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6511lb # [ 170.958380] niks3-server[1166]: 2026/09/23 04:46:00 INFO Signed narinfos id=61 count=16512lb # [ 170.967046] niks3-server[1166]: 2026/09/23 04:46:00 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6513worker1 # [ 170.998814] nix-grpc-daemon[1506]: time=2026-09-23T04:46:00.574Z level=INFO msg="Upload complete. (268ms)"6514lb # [ 170.977004] niks3-server[1166]: 2026/09/23 04:46:00 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6515lb # [ 170.984412] niks3-server[1166]: 2026/09/23 04:46:00 INFO Completed upload id=616516lb # [ 170.987542] niks3-server[1166]: 2026/09/23 04:46:00 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6517worker1 # [ 171.013202] nix-grpc-daemon[1403]: ts=2026-09-23T04:46:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34218 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16518worker1 # [ 171.032828] nix-daemon[835]: reaped child process 1555, status = succeeded6519client # [ 171.011829] systemd[1]: sw.service: Deactivated successfully.6520client # [ 171.016356] systemd[1]: sw.service: Consumed 185ms CPU time over 8.952s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6521worker1 # [ 171.716516] nix-grpc-daemon[1403]: ts=2026-09-23T04:46:01Z level=info event=signal_exit6522client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6523client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6524client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6525worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6526worker1 # [ 173.719000] nix-grpc-daemon[1403]: ts=2026-09-23T04:46:03Z level=info event=scheduler_restarting6527worker2 # [ 173.678079] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:418546528worker2 # [ 173.683050] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:03Z level=info event=worker_gone addr=192.168.1.3:500516529worker1 # [ 173.745499] nix-grpc-daemon[1403]: ts=2026-09-23T04:46:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6530worker1 # [ 173.772662] nix-daemon[835]: reaped child process 1460, status = succeeded6531worker1 # [ 174.571817] nix-grpc-daemon[1403]: E0000 00:00:1790138764.147458 1449 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6532worker1 # [ 174.588871] nix-grpc-daemon[1403]: E0000 00:00:1790138764.164533 1449 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6533worker1 # [ 175.747387] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6534worker1 # [ 175.753602] systemd[1]: nix-grpc-daemon.service: Consumed 874ms CPU time over 28.201s wall clock time, 35.6M memory peak, 21.4K incoming IP traffic, 28K outgoing IP traffic.6535worker1 # [ 175.774185] systemd[1]: Starting Nix worker-protocol over gRPC...6536worker1 # [ 175.923915] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6537worker1 # [ 175.926384] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6538worker1 # [ 175.930194] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16539worker1 # [ 175.936052] nix-grpc-daemon[1616]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6540worker1 # [ 175.937539] nix-grpc-daemon[1616]: E0000 00:00:1790138765.510388 1616 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6541worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 4.19 seconds)6542lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6543lb: must succeed: curl -sf localhost:9901/clusters6544worker1 # [ 175.980133] nix-grpc-daemon[1616]: E0000 00:00:1790138765.516708 1616 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6545worker1 # [ 175.982513] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6546worker1 # [ 175.986723] nix-grpc-daemon[1616]: E0000 00:00:1790138765.524481 1656 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6547worker1 # [ 175.989864] nix-grpc-daemon[1616]: E0000 00:00:1790138765.540751 1656 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6548worker1 # [ 175.992391] nix-grpc-daemon[1616]: E0000 00:00:1790138765.540937 1656 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6549worker1 # [ 175.994757] systemd[1]: Started Nix worker-protocol over gRPC.6550worker1 # [ 175.995581] nix-grpc-daemon[1616]: E0000 00:00:1790138765.561632 1656 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6551worker1 # [ 176.005542] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:05Z level=info event=scheduler_connected addr=lb:500516552worker2 # [ 175.966669] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:418546553worker2 # [ 175.972187] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06554lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6555lb: must succeed: curl -sf localhost:9901/clusters6556lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6557lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)6558lb: must succeed: curl -sf localhost:9901/clusters6559lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6560worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6561worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6562(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.20 seconds)6563subtest: queue gauges show a burst while every build blocks6564lb: must succeed: curl -sf localhost:9901/clusters6565lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6566lb: must succeed: curl -sf localhost:9901/clusters6567lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6568worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6569worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6570client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6571client # [ 177.425849] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6572client # Running as unit: burst.service; invocation ID: 8047e3d7945c4bd08acfc1d2d80be0436573client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6574lb: must succeed: curl -sf localhost:9901/clusters6575lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6576lb: must succeed: curl -sf localhost:9901/clusters6577lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6578worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6579worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6580worker1 # [ 177.666826] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:609646581worker1 # [ 177.681414] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6582client # [ 177.665269] nix[1552]: don't know how to build these paths:6583client # [ 177.666841] nix[1552]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6584client # [ 177.669455] nix[1552]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6585client # [ 177.671156] nix[1552]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6586client # [ 177.674063] nix[1552]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6587worker2 # [ 177.680102] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06588worker2 # [ 177.680449] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.4:50051 clients=16589worker2 # [ 177.680593] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06590worker2 # [ 177.680709] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.3:50051 clients=16591worker2 # [ 177.680862] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06592worker2 # [ 177.680991] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06593worker2 # [ 177.723575] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746594worker1 # [ 177.765417] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646595worker2 # [ 177.739910] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418746596worker1 # [ 177.782406] nix-grpc-daemon[1616]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 1/5)6597client # [ 177.760296] nix[1552]: copying 1 paths to 'lb:50051'...6598worker2 # [ 177.753990] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418746599client # [ 177.762558] nix[1552]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6600worker1 # [ 177.802649] nix-grpc-daemon[1616]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 2/5)6601worker2 # [ 177.770678] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6602worker1 # [ 177.887498] nix-grpc-daemon[1616]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 130 ms (attempt 3/5)6603worker2 # [ 177.925445] nix-daemon[859]: accepted connection from pid 1981, user nix-grpc-daemon (trusted)6604worker2 # [ 177.942392] nix-daemon[859]: reaped child process 1984, status = succeeded6605lb # [ 177.967306] niks3-server[1166]: 2026/09/23 04:46:07 INFO Received uploads request method=POST path=/api/pending_closures6606worker2 # [ 177.959841] nix-grpc-daemon[1801]: time=2026-09-23T04:46:07.527Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6607worker2 # [ 177.961906] nix-grpc-daemon[1801]: time=2026-09-23T04:46:07.529Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6608worker1 # [ 178.019666] nix-grpc-daemon[1616]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 644 ms (attempt 4/5)6609worker2 # [ 177.993069] nix-grpc-daemon[1801]: time=2026-09-23T04:46:07.560Z level=INFO msg="Uploading 1 narinfos"6610lb # [ 178.001201] niks3-server[1166]: 2026/09/23 04:46:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6611lb # [ 178.014410] niks3-server[1166]: 2026/09/23 04:46:07 INFO Signed narinfos id=62 count=16612lb # [ 178.024340] niks3-server[1166]: 2026/09/23 04:46:07 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6613lb # [ 178.035139] niks3-server[1166]: 2026/09/23 04:46:07 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6614lb # [ 178.053886] niks3-server[1166]: 2026/09/23 04:46:07 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6615lb # [ 178.058923] niks3-server[1166]: 2026/09/23 04:46:07 INFO Completed upload id=626616worker2 # [ 178.041870] nix-grpc-daemon[1801]: time=2026-09-23T04:46:07.609Z level=INFO msg="Upload complete. (254ms)"6617lb # [ 178.065791] niks3-server[1166]: 2026/09/23 04:46:07 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6618worker2 # [ 178.052349] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=4166619worker2 # [ 178.071905] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746620worker2 # [ 178.079523] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6621worker2 # [ 178.085856] nix-daemon[859]: reaped child process 1979, status = succeeded6622client # [ 178.104537] nix[1552]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6623worker2 # [ 178.099026] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6624lb: must succeed: curl -sf localhost:9901/clusters6625worker1 # [ 178.675917] nix-grpc-daemon[1616]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6626worker1 # [ 178.680173] nix-grpc-daemon[1616]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6627lb # [ 178.665800] envoy[897]: conn peer=[2001:db8:1::1]:60916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9621 tx=4382 ms=53446628lb # [ 178.674886] envoy[897]: conn peer=[2001:db8:1::1]:60912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=1517 ms=87396629lb # [ 178.686433] envoy[897]: conn peer=[2001:db8:1::3]:48682 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1020 tx=482 ms=259836630worker1 # [ 178.719419] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609646631client # [ 178.701158] nix[1552]: copying 1 paths to 'lb:50051'...6632worker1 # [ 178.740973] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609646633client # [ 178.706919] nix[1552]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6634worker1 # [ 178.757135] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6635lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)6636lb: must succeed: curl -sf localhost:9901/clusters6637worker1 # [ 178.794577] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:08Z level=info event=niks3_push_started pid=16776638lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6639worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6640worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6641worker1 # [ 178.950073] nix-grpc-daemon[1677]: time=2026-09-23T04:46:08.525Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6642worker1 # [ 179.075861] nix-daemon[835]: accepted connection from pid 1694, user nix-grpc-daemon (trusted)6643worker1 # [ 179.095661] nix-daemon[835]: reaped child process 1697, status = succeeded6644lb # [ 179.070481] niks3-server[1166]: 2026/09/23 04:46:08 INFO Received uploads request method=POST path=/api/pending_closures6645worker1 # [ 179.104216] nix-grpc-daemon[1677]: time=2026-09-23T04:46:08.680Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6646worker1 # [ 179.105734] nix-grpc-daemon[1677]: time=2026-09-23T04:46:08.680Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6647lb # [ 179.111660] niks3-server[1166]: 2026/09/23 04:46:08 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6648worker1 # [ 179.147699] nix-grpc-daemon[1677]: time=2026-09-23T04:46:08.722Z level=INFO msg="Uploading 1 narinfos"6649lb # [ 179.128168] niks3-server[1166]: 2026/09/23 04:46:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6650lb # [ 179.139677] niks3-server[1166]: 2026/09/23 04:46:08 INFO Signed narinfos id=63 count=16651lb # [ 179.150513] niks3-server[1166]: 2026/09/23 04:46:08 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6652worker1 # [ 179.181424] nix-grpc-daemon[1677]: time=2026-09-23T04:46:08.757Z level=INFO msg="Upload complete. (226ms)"6653worker2 # [ 179.141832] nix-daemon[859]: reaped child process 1988, status = succeeded6654lb # [ 179.159101] niks3-server[1166]: 2026/09/23 04:46:08 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6655lb # [ 179.164520] niks3-server[1166]: 2026/09/23 04:46:08 INFO Completed upload id=636656lb # [ 179.168424] niks3-server[1166]: 2026/09/23 04:46:08 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6657worker1 # [ 179.192312] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=4166658worker1 # [ 179.202961] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646659worker1 # [ 179.211569] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6660worker1 # [ 179.214149] nix-daemon[835]: reaped child process 1675, status = succeeded6661client # [ 179.189496] nix[1552]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6662worker1 # [ 179.227348] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6663worker2 # [ 179.264787] nix-daemon[859]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)6664worker2 # [ 179.285568] nix-daemon[859]: reaped child process 2015, status = succeeded6665lb # [ 179.300392] niks3-server[1166]: 2026/09/23 04:46:08 INFO Received uploads request method=POST path=/api/pending_closures6666worker2 # [ 179.295143] nix-grpc-daemon[1801]: time=2026-09-23T04:46:08.860Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6667worker2 # [ 179.296861] nix-grpc-daemon[1801]: time=2026-09-23T04:46:08.860Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6668lb # [ 179.337113] niks3-server[1166]: 2026/09/23 04:46:08 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6669lb # [ 179.350567] niks3-server[1166]: 2026/09/23 04:46:08 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6670worker2 # [ 179.341000] nix-grpc-daemon[1801]: time=2026-09-23T04:46:08.908Z level=INFO msg="Uploading 1 narinfos"6671lb # [ 179.361490] niks3-server[1166]: 2026/09/23 04:46:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6672lb # [ 179.369546] niks3-server[1166]: 2026/09/23 04:46:08 INFO Signed narinfos id=64 count=16673lb # [ 179.371783] niks3-server[1166]: 2026/09/23 04:46:08 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6674worker2 # [ 179.366562] nix-grpc-daemon[1801]: time=2026-09-23T04:46:08.934Z level=INFO msg="Upload complete. (254ms)"6675lb # [ 179.384594] niks3-server[1166]: 2026/09/23 04:46:08 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6676lb # [ 179.387992] niks3-server[1166]: 2026/09/23 04:46:08 INFO Completed upload id=646677lb # [ 179.392079] niks3-server[1166]: 2026/09/23 04:46:08 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6678worker2 # [ 179.380859] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=16679worker2 # [ 179.395880] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:08Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16680worker2 # [ 179.402359] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746681worker2 # [ 179.407875] nix-daemon[859]: reaped child process 1986, status = succeeded6682worker2 # [ 179.411670] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418746683client # [ 179.427724] nix[1552]: copying 1 paths to 'lb:50051'...6684worker2 # [ 179.420614] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418746685client # [ 179.430390] nix[1552]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6686worker2 # [ 179.431672] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6687worker2 # [ 179.562100] nix-daemon[859]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6688worker2 # [ 179.581470] nix-daemon[859]: reaped child process 2022, status = succeeded6689lb # [ 179.596268] niks3-server[1166]: 2026/09/23 04:46:09 INFO Received uploads request method=POST path=/api/pending_closures6690worker2 # [ 179.590671] nix-grpc-daemon[1801]: time=2026-09-23T04:46:09.158Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6691worker2 # [ 179.592255] nix-grpc-daemon[1801]: time=2026-09-23T04:46:09.158Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6692lb # [ 179.622752] niks3-server[1166]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6693worker2 # [ 179.616073] nix-grpc-daemon[1801]: time=2026-09-23T04:46:09.183Z level=INFO msg="Uploading 1 narinfos"6694lb # [ 179.632688] niks3-server[1166]: 2026/09/23 04:46:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6695lb # [ 179.635853] niks3-server[1166]: 2026/09/23 04:46:09 INFO Signed narinfos id=65 count=16696lb # [ 179.643142] niks3-server[1166]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6697lb # [ 179.654462] niks3-server[1166]: 2026/09/23 04:46:09 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6698worker2 # [ 179.641696] nix-grpc-daemon[1801]: time=2026-09-23T04:46:09.209Z level=INFO msg="Upload complete. (200ms)"6699lb # [ 179.660541] niks3-server[1166]: 2026/09/23 04:46:09 INFO Completed upload id=656700lb # [ 179.663493] niks3-server[1166]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6701worker2 # [ 179.648230] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=4166702worker2 # [ 179.659443] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418746703worker2 # [ 179.667628] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6704worker2 # [ 179.670432] nix-daemon[859]: reaped child process 2017, status = succeeded6705worker2 # [ 179.681913] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6706client # [ 179.687190] nix[1552]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6707worker1 # [ 180.253333] nix-daemon[835]: reaped child process 1701, status = succeeded6708worker1 # [ 180.354630] nix-daemon[835]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6709worker1 # [ 180.375553] nix-daemon[835]: reaped child process 1725, status = succeeded6710lb # [ 180.355483] niks3-server[1166]: 2026/09/23 04:46:09 INFO Received uploads request method=POST path=/api/pending_closures6711worker1 # [ 180.390843] nix-grpc-daemon[1677]: time=2026-09-23T04:46:09.964Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6712worker1 # [ 180.393346] nix-grpc-daemon[1677]: time=2026-09-23T04:46:09.964Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6713lb # [ 180.392967] niks3-server[1166]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6714lb # [ 180.405589] niks3-server[1166]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6715worker1 # [ 180.449344] nix-grpc-daemon[1677]: time=2026-09-23T04:46:10.024Z level=INFO msg="Uploading 1 narinfos"6716lb # [ 180.417995] niks3-server[1166]: 2026/09/23 04:46:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6717lb # [ 180.429194] niks3-server[1166]: 2026/09/23 04:46:09 INFO Signed narinfos id=66 count=16718lb # [ 180.431839] niks3-server[1166]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6719lb # [ 180.444905] niks3-server[1166]: 2026/09/23 04:46:10 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6720lb # [ 180.450668] niks3-server[1166]: 2026/09/23 04:46:10 INFO Completed upload id=666721worker1 # [ 180.475093] nix-grpc-daemon[1677]: time=2026-09-23T04:46:10.050Z level=INFO msg="Upload complete. (237ms)"6722lb # [ 180.454056] niks3-server[1166]: 2026/09/23 04:46:10 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6723worker2 # [ 180.441521] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:10Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16724worker1 # [ 180.489374] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=16725worker1 # [ 180.505489] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646726worker1 # [ 180.510739] nix-daemon[835]: reaped child process 1699, status = succeeded6727worker1 # [ 180.518139] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609646728worker1 # [ 180.530047] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609646729client # [ 180.497001] nix[1552]: copying 1 paths to 'lb:50051'...6730client # [ 180.497266] nix[1552]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6731worker1 # [ 180.540123] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6732worker1 # [ 180.675828] nix-daemon[835]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6733worker1 # [ 180.695094] nix-daemon[835]: reaped child process 1732, status = succeeded6734lb # [ 180.674310] niks3-server[1166]: 2026/09/23 04:46:10 INFO Received uploads request method=POST path=/api/pending_closures6735worker1 # [ 180.707145] nix-grpc-daemon[1677]: time=2026-09-23T04:46:10.282Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6736worker1 # [ 180.708864] nix-grpc-daemon[1677]: time=2026-09-23T04:46:10.282Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6737worker1 # [ 180.728182] nix-grpc-daemon[1677]: time=2026-09-23T04:46:10.303Z level=INFO msg="Uploading 1 narinfos"6738lb # [ 180.703375] niks3-server[1166]: 2026/09/23 04:46:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6739lb # [ 180.718198] niks3-server[1166]: 2026/09/23 04:46:10 INFO Signed narinfos id=67 count=16740lb # [ 180.726256] niks3-server[1166]: 2026/09/23 04:46:10 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6741lb # [ 180.740979] niks3-server[1166]: 2026/09/23 04:46:10 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6742worker1 # [ 180.773602] nix-grpc-daemon[1677]: time=2026-09-23T04:46:10.349Z level=INFO msg="Upload complete. (221ms)"6743lb # [ 180.751593] niks3-server[1166]: 2026/09/23 04:46:10 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6744lb # [ 180.756339] niks3-server[1166]: 2026/09/23 04:46:10 INFO Completed upload id=676745lb # [ 180.759796] niks3-server[1166]: 2026/09/23 04:46:10 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6746worker1 # [ 180.785142] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=4166747worker1 # [ 180.798954] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646748worker1 # [ 180.805120] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6749worker1 # [ 180.809931] nix-daemon[835]: reaped child process 1727, status = succeeded6750client # [ 180.784504] nix[1552]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6751worker1 # [ 180.821925] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6752worker2 # [ 181.522122] nix-daemon[859]: reaped child process 2026, status = succeeded6753worker2 # [ 181.620259] nix-daemon[859]: accepted connection from pid 2051, user nix-grpc-daemon (trusted)6754worker2 # [ 181.637877] nix-daemon[859]: reaped child process 2054, status = succeeded6755lb # [ 181.661847] niks3-server[1166]: 2026/09/23 04:46:11 INFO Received uploads request method=POST path=/api/pending_closures6756worker2 # [ 181.652365] nix-grpc-daemon[1801]: time=2026-09-23T04:46:11.220Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6757worker2 # [ 181.656993] nix-grpc-daemon[1801]: time=2026-09-23T04:46:11.220Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6758lb # [ 181.704773] niks3-server[1166]: 2026/09/23 04:46:11 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6759lb # [ 181.717807] niks3-server[1166]: 2026/09/23 04:46:11 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6760worker2 # [ 181.709230] nix-grpc-daemon[1801]: time=2026-09-23T04:46:11.277Z level=INFO msg="Uploading 1 narinfos"6761lb # [ 181.730662] niks3-server[1166]: 2026/09/23 04:46:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6762lb # [ 181.738179] niks3-server[1166]: 2026/09/23 04:46:11 INFO Signed narinfos id=68 count=16763lb # [ 181.742765] niks3-server[1166]: 2026/09/23 04:46:11 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6764lb # [ 181.750926] niks3-server[1166]: 2026/09/23 04:46:11 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6765lb # [ 181.755760] niks3-server[1166]: 2026/09/23 04:46:11 INFO Completed upload id=686766worker2 # [ 181.738301] nix-grpc-daemon[1801]: time=2026-09-23T04:46:11.306Z level=INFO msg="Upload complete. (244ms)"6767lb # [ 181.759411] niks3-server[1166]: 2026/09/23 04:46:11 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6768worker2 # [ 181.753046] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=16769worker2 # [ 181.767768] nix-daemon[859]: reaped child process 2024, status = succeeded6770worker1 # [ 182.647353] nix-daemon[835]: reaped child process 1736, status = succeeded6771worker1 # [ 182.746705] nix-daemon[835]: accepted connection from pid 1761, user nix-grpc-daemon (trusted)6772worker1 # [ 182.767242] nix-daemon[835]: reaped child process 1764, status = succeeded6773lb # [ 182.746432] niks3-server[1166]: 2026/09/23 04:46:12 INFO Received uploads request method=POST path=/api/pending_closures6774worker1 # [ 182.781347] nix-grpc-daemon[1677]: time=2026-09-23T04:46:12.355Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6775worker1 # [ 182.782890] nix-grpc-daemon[1677]: time=2026-09-23T04:46:12.356Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6776lb # [ 182.785646] niks3-server[1166]: 2026/09/23 04:46:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6777worker1 # [ 182.820596] nix-grpc-daemon[1677]: time=2026-09-23T04:46:12.396Z level=INFO msg="Uploading 1 narinfos"6778lb # [ 182.802413] niks3-server[1166]: 2026/09/23 04:46:12 INFO Signed narinfos id=69 count=16779lb # [ 182.810968] niks3-server[1166]: 2026/09/23 04:46:12 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6780lb # [ 182.819540] niks3-server[1166]: 2026/09/23 04:46:12 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6781lb # [ 182.831386] niks3-server[1166]: 2026/09/23 04:46:12 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6782worker1 # [ 182.861212] nix-grpc-daemon[1677]: time=2026-09-23T04:46:12.437Z level=INFO msg="Upload complete. (229ms)"6783lb # [ 182.836741] niks3-server[1166]: 2026/09/23 04:46:12 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6784lb # [ 182.841241] niks3-server[1166]: 2026/09/23 04:46:12 INFO Completed upload id=696785lb # [ 182.844483] niks3-server[1166]: 2026/09/23 04:46:12 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6786worker1 # [ 182.870606] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=16787worker1 # [ 182.886610] nix-daemon[835]: reaped child process 1734, status = succeeded6788client # [ 182.874536] systemd[1]: burst.service: Deactivated successfully.6789client # [ 182.880311] systemd[1]: burst.service: Consumed 223ms CPU time over 5.450s wall clock time, 53.6M memory peak, 14K incoming IP traffic, 18.7K outgoing IP traffic.6790client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6791client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6792lb: must succeed: curl -sf localhost:9901/clusters6793lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6794lb: must succeed: curl -sf localhost:9901/clusters6795lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6796worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6797worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6798(finished: subtest: queue gauges show a burst while every build blocks, in 6.94 seconds)6799subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6800client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16801worker1 # [ 184.397562] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:609646802worker1 # [ 184.405224] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609646803worker2 # [ 184.405370] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:13Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06804worker2 # [ 184.411528] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:13Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16805worker2 # [ 184.417814] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:13Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06806worker2 # [ 184.423549] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:13Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06807worker1 # [ 184.500577] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646808worker1 # [ 184.515601] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609646809worker1 # [ 184.529668] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609646810worker1 # [ 184.544765] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6811worker1 # [ 184.715943] nix-daemon[835]: accepted connection from pid 1772, user nix-grpc-daemon (trusted)6812worker1 # [ 184.733458] nix-daemon[835]: reaped child process 1775, status = succeeded6813lb # [ 184.717788] niks3-server[1166]: 2026/09/23 04:46:14 INFO Received uploads request method=POST path=/api/pending_closures6814worker1 # [ 184.749931] nix-grpc-daemon[1677]: time=2026-09-23T04:46:14.325Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6815worker1 # [ 184.751476] nix-grpc-daemon[1677]: time=2026-09-23T04:46:14.325Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6816lb # [ 184.746720] niks3-server[1166]: 2026/09/23 04:46:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6817worker1 # [ 184.779565] nix-grpc-daemon[1677]: time=2026-09-23T04:46:14.355Z level=INFO msg="Uploading 1 narinfos"6818lb # [ 184.763302] niks3-server[1166]: 2026/09/23 04:46:14 INFO Signed narinfos id=70 count=16819lb # [ 184.768806] niks3-server[1166]: 2026/09/23 04:46:14 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6820worker1 # [ 184.806696] nix-grpc-daemon[1677]: time=2026-09-23T04:46:14.382Z level=INFO msg="Upload complete. (245ms)"6821lb # [ 184.783004] niks3-server[1166]: 2026/09/23 04:46:14 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6822lb # [ 184.792214] niks3-server[1166]: 2026/09/23 04:46:14 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6823worker1 # [ 184.815390] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=5126824lb # [ 184.796580] niks3-server[1166]: 2026/09/23 04:46:14 INFO Completed upload id=706825lb # [ 184.799368] niks3-server[1166]: 2026/09/23 04:46:14 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6826worker1 # [ 184.831140] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646827worker1 # [ 184.840215] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6828worker1 # [ 184.843346] nix-daemon[835]: reaped child process 1770, status = succeeded6829worker1 # [ 184.856216] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6830worker1 # [ 185.893014] nix-daemon[835]: reaped child process 1779, status = succeeded6831worker1 # [ 185.991123] nix-daemon[835]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6832worker1 # [ 186.010592] nix-daemon[835]: reaped child process 1798, status = succeeded6833lb # [ 185.990915] niks3-server[1166]: 2026/09/23 04:46:15 INFO Received uploads request method=POST path=/api/pending_closures6834worker1 # [ 186.028386] nix-grpc-daemon[1677]: time=2026-09-23T04:46:15.602Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6835worker1 # [ 186.029874] nix-grpc-daemon[1677]: time=2026-09-23T04:46:15.603Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6836lb # [ 186.030103] niks3-server[1166]: 2026/09/23 04:46:15 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6837lb # [ 186.043576] niks3-server[1166]: 2026/09/23 04:46:15 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6838worker1 # [ 186.076288] nix-grpc-daemon[1677]: time=2026-09-23T04:46:15.651Z level=INFO msg="Uploading 1 narinfos"6839lb # [ 186.055633] niks3-server[1166]: 2026/09/23 04:46:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6840lb # [ 186.061903] niks3-server[1166]: 2026/09/23 04:46:15 INFO Signed narinfos id=71 count=16841lb # [ 186.066127] niks3-server[1166]: 2026/09/23 04:46:15 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6842lb # [ 186.073238] niks3-server[1166]: 2026/09/23 04:46:15 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6843lb # [ 186.078279] niks3-server[1166]: 2026/09/23 04:46:15 INFO Completed upload id=716844worker1 # [ 186.102786] nix-grpc-daemon[1677]: time=2026-09-23T04:46:15.678Z level=INFO msg="Upload complete. (237ms)"6845lb # [ 186.081233] niks3-server[1166]: 2026/09/23 04:46:15 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6846worker2 # [ 186.074054] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:15Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16847worker1 # [ 186.115660] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=16848worker2 # [ 186.081608] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:15Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06849worker1 # [ 186.132315] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646850worker1 # [ 186.141932] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609646851worker1 # [ 186.147660] nix-daemon[835]: reaped child process 1777, status = succeeded6852worker1 # [ 186.154471] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609646853worker1 # [ 186.167622] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6854worker1 # [ 186.294577] nix-daemon[835]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6855worker1 # [ 186.316485] nix-daemon[835]: reaped child process 1805, status = succeeded6856lb # [ 186.290320] niks3-server[1166]: 2026/09/23 04:46:15 INFO Received uploads request method=POST path=/api/pending_closures6857worker1 # [ 186.323684] nix-grpc-daemon[1677]: time=2026-09-23T04:46:15.899Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6858worker1 # [ 186.325490] nix-grpc-daemon[1677]: time=2026-09-23T04:46:15.899Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6859lb # [ 186.317898] niks3-server[1166]: 2026/09/23 04:46:15 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6860worker1 # [ 186.351903] nix-grpc-daemon[1677]: time=2026-09-23T04:46:15.927Z level=INFO msg="Uploading 1 narinfos"6861lb # [ 186.329904] niks3-server[1166]: 2026/09/23 04:46:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6862lb # [ 186.339945] niks3-server[1166]: 2026/09/23 04:46:15 INFO Signed narinfos id=72 count=16863lb # [ 186.344515] niks3-server[1166]: 2026/09/23 04:46:15 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6864worker1 # [ 186.378658] nix-grpc-daemon[1677]: time=2026-09-23T04:46:15.954Z level=INFO msg="Upload complete. (202ms)"6865lb # [ 186.354944] niks3-server[1166]: 2026/09/23 04:46:15 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6866lb # [ 186.363337] niks3-server[1166]: 2026/09/23 04:46:15 INFO Completed upload id=726867worker1 # [ 186.387411] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=5126868lb # [ 186.367197] niks3-server[1166]: 2026/09/23 04:46:15 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6869worker1 # [ 186.399758] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646870worker1 # [ 186.407495] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6871worker1 # [ 186.409667] nix-daemon[835]: reaped child process 1800, status = succeeded6872worker1 # [ 186.420750] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6873worker1 # [ 187.892556] nix-daemon[835]: reaped child process 1809, status = succeeded6874worker1 # [ 187.998605] nix-daemon[835]: accepted connection from pid 1825, user nix-grpc-daemon (trusted)6875worker1 # [ 188.018041] nix-daemon[835]: reaped child process 1828, status = succeeded6876lb # [ 187.998900] niks3-server[1166]: 2026/09/23 04:46:17 INFO Received uploads request method=POST path=/api/pending_closures6877worker1 # [ 188.035434] nix-grpc-daemon[1677]: time=2026-09-23T04:46:17.608Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6878worker1 # [ 188.037199] nix-grpc-daemon[1677]: time=2026-09-23T04:46:17.608Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6879lb # [ 188.039367] niks3-server[1166]: 2026/09/23 04:46:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6880worker1 # [ 188.073009] nix-grpc-daemon[1677]: time=2026-09-23T04:46:17.648Z level=INFO msg="Uploading 1 narinfos"6881lb # [ 188.054344] niks3-server[1166]: 2026/09/23 04:46:17 INFO Signed narinfos id=73 count=16882lb # [ 188.063802] niks3-server[1166]: 2026/09/23 04:46:17 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6883lb # [ 188.071530] niks3-server[1166]: 2026/09/23 04:46:17 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6884lb # [ 188.082379] niks3-server[1166]: 2026/09/23 04:46:17 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6885worker1 # [ 188.111446] nix-grpc-daemon[1677]: time=2026-09-23T04:46:17.687Z level=INFO msg="Upload complete. (281ms)"6886lb # [ 188.088615] niks3-server[1166]: 2026/09/23 04:46:17 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6887lb # [ 188.092579] niks3-server[1166]: 2026/09/23 04:46:17 INFO Completed upload id=736888lb # [ 188.096290] niks3-server[1166]: 2026/09/23 04:46:17 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6889worker2 # [ 188.081342] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:17Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16890worker1 # [ 188.125097] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=16891worker2 # [ 188.094218] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:17Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06892worker1 # [ 188.147547] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646893worker1 # [ 188.154708] nix-daemon[835]: reaped child process 1807, status = succeeded6894worker1 # [ 188.159288] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609646895worker1 # [ 188.168762] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609646896worker1 # [ 188.180793] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6897worker1 # [ 188.319795] nix-daemon[835]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)6898worker1 # [ 188.336829] nix-daemon[835]: reaped child process 1835, status = succeeded6899lb # [ 188.317511] niks3-server[1166]: 2026/09/23 04:46:17 INFO Received uploads request method=POST path=/api/pending_closures6900worker1 # [ 188.350636] nix-grpc-daemon[1677]: time=2026-09-23T04:46:17.926Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6901worker1 # [ 188.352573] nix-grpc-daemon[1677]: time=2026-09-23T04:46:17.926Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6902lb # [ 188.347369] niks3-server[1166]: 2026/09/23 04:46:17 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6903worker1 # [ 188.382880] nix-grpc-daemon[1677]: time=2026-09-23T04:46:17.958Z level=INFO msg="Uploading 1 narinfos"6904lb # [ 188.362479] niks3-server[1166]: 2026/09/23 04:46:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6905lb # [ 188.372815] niks3-server[1166]: 2026/09/23 04:46:17 INFO Signed narinfos id=74 count=16906lb # [ 188.378006] niks3-server[1166]: 2026/09/23 04:46:17 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6907lb # [ 188.386094] niks3-server[1166]: 2026/09/23 04:46:17 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6908lb # [ 188.391250] niks3-server[1166]: 2026/09/23 04:46:17 INFO Completed upload id=746909worker1 # [ 188.414939] nix-grpc-daemon[1677]: time=2026-09-23T04:46:17.990Z level=INFO msg="Upload complete. (221ms)"6910lb # [ 188.393882] niks3-server[1166]: 2026/09/23 04:46:17 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6911worker1 # [ 188.425115] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=5126912worker1 # [ 188.438791] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646913worker1 # [ 188.447063] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6914worker1 # [ 188.452160] nix-daemon[835]: reaped child process 1830, status = succeeded6915worker1 # [ 188.462137] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6916lb # [ 188.665754] envoy[897]: conn peer=[2001:db8:1::1]:44126 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9697 tx=5196 ms=52286917worker1 # [ 189.368715] nix-daemon[835]: reaped child process 1839, status = succeeded6918worker1 # [ 189.443073] nix-daemon[835]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6919worker1 # [ 189.463031] nix-daemon[835]: reaped child process 1858, status = succeeded6920lb # [ 189.443735] niks3-server[1166]: 2026/09/23 04:46:19 INFO Received uploads request method=POST path=/api/pending_closures6921worker1 # [ 189.480832] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.053Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6922worker1 # [ 189.482492] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.053Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6923lb # [ 189.483451] niks3-server[1166]: 2026/09/23 04:46:19 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6924worker1 # [ 189.521140] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.096Z level=INFO msg="Uploading 1 narinfos"6925lb # [ 189.496995] niks3-server[1166]: 2026/09/23 04:46:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6926lb # [ 189.507051] niks3-server[1166]: 2026/09/23 04:46:19 INFO Signed narinfos id=75 count=16927lb # [ 189.513480] niks3-server[1166]: 2026/09/23 04:46:19 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6928worker1 # [ 189.549154] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.124Z level=INFO msg="Upload complete. (276ms)"6929lb # [ 189.525073] niks3-server[1166]: 2026/09/23 04:46:19 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6930lb # [ 189.529854] niks3-server[1166]: 2026/09/23 04:46:19 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6931lb # [ 189.534350] niks3-server[1166]: 2026/09/23 04:46:19 INFO Completed upload id=756932lb # [ 189.537067] niks3-server[1166]: 2026/09/23 04:46:19 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6933worker2 # [ 189.519315] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:19Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16934worker1 # [ 189.561042] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=16935worker1 # [ 189.579143] nix-daemon[835]: reaped child process 1837, status = succeeded6936worker1 # [ 189.606526] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646937worker2 # [ 189.568355] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:19Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06938worker1 # [ 189.622991] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609646939worker1 # [ 189.636762] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.212Z level=INFO msg="All 1 paths already cached"6940worker1 # [ 189.646130] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609646941worker1 # [ 189.660088] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6942worker1 # [ 189.806387] nix-daemon[835]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6943worker1 # [ 189.825129] nix-daemon[835]: reaped child process 1865, status = succeeded6944lb # [ 189.805420] niks3-server[1166]: 2026/09/23 04:46:19 INFO Received uploads request method=POST path=/api/pending_closures6945worker1 # [ 189.838753] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.414Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6946worker1 # [ 189.840512] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.414Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6947lb # [ 189.835874] niks3-server[1166]: 2026/09/23 04:46:19 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6948worker1 # [ 189.870202] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.445Z level=INFO msg="Uploading 1 narinfos"6949lb # [ 189.849754] niks3-server[1166]: 2026/09/23 04:46:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6950lb # [ 189.860645] niks3-server[1166]: 2026/09/23 04:46:19 INFO Signed narinfos id=76 count=16951worker1 # [ 189.893242] nix-grpc-daemon[1677]: time=2026-09-23T04:46:19.468Z level=INFO msg="Upload complete. (217ms)"6952lb # [ 189.869404] niks3-server[1166]: 2026/09/23 04:46:19 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6953lb # [ 189.879708] niks3-server[1166]: 2026/09/23 04:46:19 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6954worker1 # [ 189.903118] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=6246955lb # [ 189.883800] niks3-server[1166]: 2026/09/23 04:46:19 INFO Completed upload id=766956lb # [ 189.886650] niks3-server[1166]: 2026/09/23 04:46:19 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6957worker1 # [ 189.915703] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646958worker1 # [ 189.922749] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6959worker1 # [ 189.927539] nix-daemon[835]: reaped child process 1860, status = succeeded6960worker1 # [ 189.940190] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6961worker1 # [ 191.198134] nix-daemon[835]: reaped child process 1869, status = succeeded6962worker1 # [ 191.314585] nix-daemon[835]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6963worker1 # [ 191.333675] nix-daemon[835]: reaped child process 1888, status = succeeded6964lb # [ 191.315978] niks3-server[1166]: 2026/09/23 04:46:20 INFO Received uploads request method=POST path=/api/pending_closures6965worker1 # [ 191.352439] nix-grpc-daemon[1677]: time=2026-09-23T04:46:20.926Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6966worker1 # [ 191.354140] nix-grpc-daemon[1677]: time=2026-09-23T04:46:20.926Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6967lb # [ 191.357027] niks3-server[1166]: 2026/09/23 04:46:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6968worker1 # [ 191.389502] nix-grpc-daemon[1677]: time=2026-09-23T04:46:20.965Z level=INFO msg="Uploading 1 narinfos"6969lb # [ 191.368900] niks3-server[1166]: 2026/09/23 04:46:20 INFO Signed narinfos id=77 count=16970lb # [ 191.378903] niks3-server[1166]: 2026/09/23 04:46:20 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6971lb # [ 191.386406] niks3-server[1166]: 2026/09/23 04:46:20 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6972lb # [ 191.395661] niks3-server[1166]: 2026/09/23 04:46:20 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6973worker1 # [ 191.420319] nix-grpc-daemon[1677]: time=2026-09-23T04:46:20.996Z level=INFO msg="Upload complete. (267ms)"6974lb # [ 191.400590] niks3-server[1166]: 2026/09/23 04:46:20 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6975lb # [ 191.404554] niks3-server[1166]: 2026/09/23 04:46:20 INFO Completed upload id=776976lb # [ 191.407667] niks3-server[1166]: 2026/09/23 04:46:20 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6977worker2 # [ 191.389459] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:20Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16978worker1 # [ 191.432265] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=16979worker2 # [ 191.402608] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:20Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06980worker1 # [ 191.452637] nix-daemon[835]: reaped child process 1867, status = succeeded6981worker1 # [ 191.484887] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609646982worker1 # [ 191.500933] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609646983worker1 # [ 191.516188] nix-grpc-daemon[1677]: time=2026-09-23T04:46:21.091Z level=INFO msg="All 1 paths already cached"6984worker1 # [ 191.525834] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609646985worker1 # [ 191.538919] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6986worker1 # [ 191.690865] nix-daemon[835]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6987worker1 # [ 191.711938] nix-daemon[835]: reaped child process 1895, status = succeeded6988lb # [ 191.690544] niks3-server[1166]: 2026/09/23 04:46:21 INFO Received uploads request method=POST path=/api/pending_closures6989worker1 # [ 191.723842] nix-grpc-daemon[1677]: time=2026-09-23T04:46:21.299Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6990worker1 # [ 191.725596] nix-grpc-daemon[1677]: time=2026-09-23T04:46:21.299Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6991lb # [ 191.722467] niks3-server[1166]: 2026/09/23 04:46:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6992worker1 # [ 191.753810] nix-grpc-daemon[1677]: time=2026-09-23T04:46:21.329Z level=INFO msg="Uploading 1 narinfos"6993lb # [ 191.735048] niks3-server[1166]: 2026/09/23 04:46:21 INFO Signed narinfos id=78 count=16994lb # [ 191.744982] niks3-server[1166]: 2026/09/23 04:46:21 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6995lb # [ 191.752701] niks3-server[1166]: 2026/09/23 04:46:21 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6996worker1 # [ 191.785278] nix-grpc-daemon[1677]: time=2026-09-23T04:46:21.361Z level=INFO msg="Upload complete. (230ms)"6997lb # [ 191.762589] niks3-server[1166]: 2026/09/23 04:46:21 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6998lb # [ 191.769443] niks3-server[1166]: 2026/09/23 04:46:21 INFO Completed upload id=786999lb # [ 191.772116] niks3-server[1166]: 2026/09/23 04:46:21 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo7000worker1 # [ 191.794809] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=6247001worker1 # [ 191.808513] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647002worker1 # [ 191.822669] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7003worker1 # [ 191.825345] nix-daemon[835]: reaped child process 1890, status = succeeded7004worker1 # [ 191.838724] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7005worker1 # [ 193.495777] nix-daemon[835]: reaped child process 1899, status = succeeded7006worker1 # [ 193.631049] nix-daemon[835]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)7007worker1 # [ 193.647720] nix-daemon[835]: reaped child process 1918, status = succeeded7008lb # [ 193.629655] niks3-server[1166]: 2026/09/23 04:46:23 INFO Received uploads request method=POST path=/api/pending_closures7009worker1 # [ 193.664414] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.239Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7010worker1 # [ 193.669898] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.239Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"7011lb # [ 193.675808] niks3-server[1166]: 2026/09/23 04:46:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7012worker1 # [ 193.707062] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.282Z level=INFO msg="Uploading 1 narinfos"7013lb # [ 193.690852] niks3-server[1166]: 2026/09/23 04:46:23 INFO Signed narinfos id=79 count=17014lb # [ 193.697398] niks3-server[1166]: 2026/09/23 04:46:23 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst7015lb # [ 193.710175] niks3-server[1166]: 2026/09/23 04:46:23 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls7016lb # [ 193.714743] niks3-server[1166]: 2026/09/23 04:46:23 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7017worker1 # [ 193.738928] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.314Z level=INFO msg="Upload complete. (289ms)"7018lb # [ 193.720438] niks3-server[1166]: 2026/09/23 04:46:23 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv7019lb # [ 193.725294] niks3-server[1166]: 2026/09/23 04:46:23 INFO Completed upload id=797020lb # [ 193.727831] niks3-server[1166]: 2026/09/23 04:46:23 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo7021worker2 # [ 193.710103] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:23Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=17022worker1 # [ 193.753988] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=17023worker2 # [ 193.724939] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:23Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=07024worker1 # [ 193.772395] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647025worker1 # [ 193.779212] nix-daemon[835]: reaped child process 1897, status = succeeded7026worker1 # [ 193.787059] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609647027worker1 # [ 193.800965] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.376Z level=INFO msg="All 1 paths already cached"7028worker1 # [ 193.810934] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609647029worker1 # [ 193.826037] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7030worker1 # [ 193.969119] nix-daemon[835]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)7031worker1 # [ 193.988856] nix-daemon[835]: reaped child process 1925, status = succeeded7032lb # [ 193.967658] niks3-server[1166]: 2026/09/23 04:46:23 INFO Received uploads request method=POST path=/api/pending_closures7033worker1 # [ 194.004989] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7034worker1 # [ 194.006621] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.580Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"7035lb # [ 194.002996] niks3-server[1166]: 2026/09/23 04:46:23 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls7036worker1 # [ 194.036204] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.611Z level=INFO msg="Uploading 1 narinfos"7037lb # [ 194.016519] niks3-server[1166]: 2026/09/23 04:46:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7038lb # [ 194.025575] niks3-server[1166]: 2026/09/23 04:46:23 INFO Signed narinfos id=80 count=17039lb # [ 194.037059] niks3-server[1166]: 2026/09/23 04:46:23 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst7040worker1 # [ 194.073241] nix-grpc-daemon[1677]: time=2026-09-23T04:46:23.649Z level=INFO msg="Upload complete. (233ms)"7041lb # [ 194.049504] niks3-server[1166]: 2026/09/23 04:46:23 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7042lb # [ 194.055384] niks3-server[1166]: 2026/09/23 04:46:23 INFO Completed upload id=807043lb # [ 194.059733] niks3-server[1166]: 2026/09/23 04:46:23 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo7044worker1 # [ 194.085123] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=6247045worker1 # [ 194.093076] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647046worker1 # [ 194.104857] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7047worker1 # [ 194.109919] nix-daemon[835]: reaped child process 1920, status = succeeded7048worker1 # [ 194.123076] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7049worker1 # [ 195.785456] nix-daemon[835]: reaped child process 1929, status = succeeded7050worker1 # [ 195.897366] nix-daemon[835]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)7051worker1 # [ 195.917896] nix-daemon[835]: reaped child process 1948, status = succeeded7052lb # [ 195.893911] niks3-server[1166]: 2026/09/23 04:46:25 INFO Received uploads request method=POST path=/api/pending_closures7053worker1 # [ 195.928488] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.503Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7054worker1 # [ 195.930022] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.503Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"7055lb # [ 195.937974] niks3-server[1166]: 2026/09/23 04:46:25 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls7056lb # [ 195.950381] niks3-server[1166]: 2026/09/23 04:46:25 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv7057worker1 # [ 195.984279] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.559Z level=INFO msg="Uploading 1 narinfos"7058lb # [ 195.963339] niks3-server[1166]: 2026/09/23 04:46:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7059lb # [ 195.973349] niks3-server[1166]: 2026/09/23 04:46:25 INFO Signed narinfos id=81 count=17060lb # [ 195.979329] niks3-server[1166]: 2026/09/23 04:46:25 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst7061worker1 # [ 196.014550] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.590Z level=INFO msg="Upload complete. (280ms)"7062lb # [ 195.989831] niks3-server[1166]: 2026/09/23 04:46:25 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7063lb # [ 195.998494] niks3-server[1166]: 2026/09/23 04:46:25 INFO Completed upload id=817064lb # [ 196.002657] niks3-server[1166]: 2026/09/23 04:46:25 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo7065worker2 # [ 195.983596] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:25Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=17066worker1 # [ 196.028817] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=17067worker2 # [ 195.997539] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:25Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=07068worker1 # [ 196.048365] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647069worker1 # [ 196.055611] nix-daemon[835]: reaped child process 1927, status = succeeded7070worker1 # [ 196.063576] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609647071worker1 # [ 196.077200] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.652Z level=INFO msg="All 2 paths already cached"7072worker1 # [ 196.086329] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609647073worker1 # [ 196.102046] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7074worker1 # [ 196.246153] nix-daemon[835]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)7075worker1 # [ 196.267258] nix-daemon[835]: reaped child process 1955, status = succeeded7076lb # [ 196.242186] niks3-server[1166]: 2026/09/23 04:46:25 INFO Received uploads request method=POST path=/api/pending_closures7077worker1 # [ 196.279865] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.855Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7078worker1 # [ 196.281580] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.855Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"7079lb # [ 196.286213] niks3-server[1166]: 2026/09/23 04:46:25 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls7080worker1 # [ 196.325435] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.901Z level=INFO msg="Uploading 1 narinfos"7081lb # [ 196.300423] niks3-server[1166]: 2026/09/23 04:46:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7082lb # [ 196.304999] niks3-server[1166]: 2026/09/23 04:46:25 INFO Signed narinfos id=82 count=17083lb # [ 196.315252] niks3-server[1166]: 2026/09/23 04:46:25 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst7084lb # [ 196.322439] niks3-server[1166]: 2026/09/23 04:46:25 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7085worker1 # [ 196.355478] nix-grpc-daemon[1677]: time=2026-09-23T04:46:25.931Z level=INFO msg="Upload complete. (243ms)"7086lb # [ 196.330579] niks3-server[1166]: 2026/09/23 04:46:25 INFO Completed upload id=827087lb # [ 196.336151] niks3-server[1166]: 2026/09/23 04:46:25 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo7088worker1 # [ 196.368580] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=7527089worker1 # [ 196.383507] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647090worker1 # [ 196.390306] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7091worker1 # [ 196.395618] nix-daemon[835]: reaped child process 1950, status = succeeded7092worker1 # [ 196.408981] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7093worker1 # [ 197.476340] nix-daemon[835]: reaped child process 1959, status = succeeded7094worker1 # [ 197.572155] nix-daemon[835]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)7095worker1 # [ 197.588703] nix-daemon[835]: reaped child process 1978, status = succeeded7096lb # [ 197.571822] niks3-server[1166]: 2026/09/23 04:46:27 INFO Received uploads request method=POST path=/api/pending_closures7097worker1 # [ 197.609859] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.183Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7098worker1 # [ 197.611519] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.183Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"7099lb # [ 197.614826] niks3-server[1166]: 2026/09/23 04:46:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7100worker1 # [ 197.648481] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.224Z level=INFO msg="Uploading 1 narinfos"7101lb # [ 197.628831] niks3-server[1166]: 2026/09/23 04:46:27 INFO Signed narinfos id=83 count=17102lb # [ 197.635158] niks3-server[1166]: 2026/09/23 04:46:27 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls7103lb # [ 197.642926] niks3-server[1166]: 2026/09/23 04:46:27 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst7104lb # [ 197.655070] niks3-server[1166]: 2026/09/23 04:46:27 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7105worker1 # [ 197.679660] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.255Z level=INFO msg="Upload complete. (257ms)"7106lb # [ 197.662886] niks3-server[1166]: 2026/09/23 04:46:27 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv7107lb # [ 197.667205] niks3-server[1166]: 2026/09/23 04:46:27 INFO Completed upload id=837108worker2 # [ 197.647285] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:27Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=17109lb # [ 197.670967] niks3-server[1166]: 2026/09/23 04:46:27 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo7110worker1 # [ 197.693251] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=17111worker1 # [ 197.710774] nix-daemon[835]: reaped child process 1957, status = succeeded7112worker1 # [ 197.737717] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647113worker1 # [ 197.752569] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609647114worker1 # [ 197.767875] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.343Z level=INFO msg="All 2 paths already cached"7115worker1 # [ 197.778135] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609647116worker1 # [ 197.791303] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7117worker1 # [ 197.945465] nix-daemon[835]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)7118worker1 # [ 197.964948] nix-daemon[835]: reaped child process 1985, status = succeeded7119lb # [ 197.944953] niks3-server[1166]: 2026/09/23 04:46:27 INFO Received uploads request method=POST path=/api/pending_closures7120worker1 # [ 197.978469] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.553Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7121worker1 # [ 197.980112] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.553Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"7122lb # [ 197.976579] niks3-server[1166]: 2026/09/23 04:46:27 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst7123worker1 # [ 198.012712] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.588Z level=INFO msg="Uploading 1 narinfos"7124lb # [ 197.993254] niks3-server[1166]: 2026/09/23 04:46:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7125lb # [ 198.003165] niks3-server[1166]: 2026/09/23 04:46:27 INFO Signed narinfos id=84 count=17126lb # [ 198.009885] niks3-server[1166]: 2026/09/23 04:46:27 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls7127lb # [ 198.017787] niks3-server[1166]: 2026/09/23 04:46:27 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7128worker1 # [ 198.048721] nix-grpc-daemon[1677]: time=2026-09-23T04:46:27.624Z level=INFO msg="Upload complete. (241ms)"7129lb # [ 198.026263] niks3-server[1166]: 2026/09/23 04:46:27 INFO Completed upload id=847130lb # [ 198.031252] niks3-server[1166]: 2026/09/23 04:46:27 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo7131worker1 # [ 198.060346] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=7527132worker1 # [ 198.072812] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647133worker1 # [ 198.087275] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7134worker1 # [ 198.090872] nix-daemon[835]: reaped child process 1980, status = succeeded7135worker1 # [ 198.106659] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7136worker1 # [ 199.570364] nix-daemon[835]: reaped child process 1989, status = succeeded7137worker1 # [ 199.703440] nix-daemon[835]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)7138worker1 # [ 199.721365] nix-daemon[835]: reaped child process 2008, status = succeeded7139lb # [ 199.703537] niks3-server[1166]: 2026/09/23 04:46:29 INFO Received uploads request method=POST path=/api/pending_closures7140worker1 # [ 199.740480] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.314Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7141worker1 # [ 199.742405] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.314Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"7142worker1 # [ 199.781354] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.356Z level=INFO msg="Uploading 1 narinfos"7143lb # [ 199.746553] niks3-server[1166]: 2026/09/23 04:46:29 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst7144lb # [ 199.761354] niks3-server[1166]: 2026/09/23 04:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7145lb # [ 199.771262] niks3-server[1166]: 2026/09/23 04:46:29 INFO Signed narinfos id=85 count=17146lb # [ 199.776196] niks3-server[1166]: 2026/09/23 04:46:29 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv7147worker1 # [ 199.810217] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.385Z level=INFO msg="Upload complete. (290ms)"7148lb # [ 199.786191] niks3-server[1166]: 2026/09/23 04:46:29 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls7149lb # [ 199.790048] niks3-server[1166]: 2026/09/23 04:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7150lb # [ 199.794892] niks3-server[1166]: 2026/09/23 04:46:29 INFO Completed upload id=857151lb # [ 199.797745] niks3-server[1166]: 2026/09/23 04:46:29 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo7152worker2 # [ 199.779045] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:29Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=17153worker1 # [ 199.823903] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=17154worker1 # [ 199.844310] nix-daemon[835]: reaped child process 1987, status = succeeded7155worker1 # [ 199.864987] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647156worker1 # [ 199.880164] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609647157worker1 # [ 199.894837] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.470Z level=INFO msg="All 2 paths already cached"7158worker1 # [ 199.905211] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:609647159worker1 # [ 199.922205] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7160worker1 # [ 200.070118] nix-daemon[835]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)7161worker1 # [ 200.089540] nix-daemon[835]: reaped child process 2015, status = succeeded7162lb # [ 200.063032] niks3-server[1166]: 2026/09/23 04:46:29 INFO Received uploads request method=POST path=/api/pending_closures7163worker1 # [ 200.096925] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.672Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7164worker1 # [ 200.098428] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.672Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"7165lb # [ 200.092675] niks3-server[1166]: 2026/09/23 04:46:29 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls7166lb # [ 200.104639] niks3-server[1166]: 2026/09/23 04:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7167worker1 # [ 200.136559] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.712Z level=INFO msg="Uploading 1 narinfos"7168lb # [ 200.117837] niks3-server[1166]: 2026/09/23 04:46:29 INFO Signed narinfos id=86 count=17169lb # [ 200.123569] niks3-server[1166]: 2026/09/23 04:46:29 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst7170worker1 # [ 200.160251] nix-grpc-daemon[1677]: time=2026-09-23T04:46:29.736Z level=INFO msg="Upload complete. (226ms)"7171lb # [ 200.136437] niks3-server[1166]: 2026/09/23 04:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7172lb # [ 200.139641] niks3-server[1166]: 2026/09/23 04:46:29 INFO Completed upload id=867173lb # [ 200.143080] niks3-server[1166]: 2026/09/23 04:46:29 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7174worker1 # [ 200.172744] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=0 paths=1 nar_bytes_in=7527175worker1 # [ 200.185437] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:609647176worker1 # [ 200.191488] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7177worker1 # [ 200.196272] nix-daemon[835]: reaped child process 2010, status = succeeded7178worker1 # [ 200.208150] nix-daemon[835]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)7179worker1 # [ 201.288720] nix-daemon[835]: reaped child process 2019, status = succeeded7180worker1 # [ 201.391877] nix-daemon[835]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7181worker1 # [ 201.411547] nix-daemon[835]: reaped child process 2038, status = succeeded7182lb # [ 201.393965] niks3-server[1166]: 2026/09/23 04:46:30 INFO Received uploads request method=POST path=/api/pending_closures7183worker1 # [ 201.432464] nix-grpc-daemon[1677]: time=2026-09-23T04:46:31.003Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7184worker1 # [ 201.434356] nix-grpc-daemon[1677]: time=2026-09-23T04:46:31.003Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7185lb # [ 201.438155] niks3-server[1166]: 2026/09/23 04:46:31 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7186worker1 # [ 201.472078] nix-grpc-daemon[1677]: time=2026-09-23T04:46:31.047Z level=INFO msg="Uploading 1 narinfos"7187lb # [ 201.452261] niks3-server[1166]: 2026/09/23 04:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7188lb # [ 201.455978] niks3-server[1166]: 2026/09/23 04:46:31 INFO Signed narinfos id=87 count=17189lb # [ 201.463658] niks3-server[1166]: 2026/09/23 04:46:31 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7190worker1 # [ 201.499974] nix-grpc-daemon[1677]: time=2026-09-23T04:46:31.075Z level=INFO msg="Upload complete. (281ms)"7191lb # [ 201.476081] niks3-server[1166]: 2026/09/23 04:46:31 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7192lb # [ 201.479365] niks3-server[1166]: 2026/09/23 04:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7193lb # [ 201.483776] niks3-server[1166]: 2026/09/23 04:46:31 INFO Completed upload id=877194lb # [ 201.486172] niks3-server[1166]: 2026/09/23 04:46:31 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7195worker1 # [ 201.512955] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60964 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=17196worker1 # [ 201.529553] nix-daemon[835]: reaped child process 2017, status = succeeded7197client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.41 seconds)7198worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7199worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7200client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7201client # [ 201.675324] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7202client # Running as unit: gpu.service; invocation ID: d7ec0d33bd4c41ecb8eb25ea1bbb41807203client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.08 seconds)7204lb: must succeed: curl -sf localhost:9901/clusters7205lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7206lb: must succeed: curl -sf localhost:9901/clusters7207lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7208worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7209worker2 # [ 201.871686] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418747210client # [ 201.885893] nix[1622]: don't know how to build these paths:7211client # [ 201.887003] nix[1622]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7212worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7213worker1 # [ 201.925660] nix-grpc-daemon[1616]: ts=2026-09-23T04:46:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:609647214worker2 # [ 201.906628] nix-grpc-daemon[1724]: ts=2026-09-23T04:46:31Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07215client # [ 201.914418] nix[1622]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7216lb: must succeed: curl -sf localhost:9901/clusters7217lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7218lb: must succeed: curl -sf localhost:9901/clusters7219lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7220worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7221worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7222client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7223lb # [ 208.666874] envoy[897]: conn peer=[2001:db8:1::1]:44902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27187 tx=12403 ms=171367224client # [ 231.921792] nix[1622]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7225client # [ 231.938063] nix[1622]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7226client # [ 231.980586] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7227client # [ 231.987227] systemd[1]: gpu.service: Failed with result 'exit-code'.7228client # [ 231.990379] systemd[1]: gpu.service: Consumed 205ms CPU time over 30.310s wall clock time, 37.5M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7229client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.16 seconds)7230client: must succeed: journalctl -u gpu -o cat7231client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7232lb: must succeed: curl -sf localhost:9901/clusters7233lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7234lb: must succeed: curl -sf localhost:9901/clusters7235lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7236worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7237worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7238(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.32 seconds)7239subtest: low disk drains a worker and builds go to the other7240worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7241worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.21 seconds)7242worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7243worker1 # [ 233.974609] nix-grpc-daemon[1616]: ts=2026-09-23T04:47:03Z level=info event=unhealthy reason=min_free7244worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)7245client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27246worker1 # [ 235.041197] nix-grpc-daemon[1616]: ts=2026-09-23T04:47:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:609647247client # don't know how to build these paths:7248client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7249worker2 # [ 235.014544] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418747250worker2 # [ 235.045626] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:04Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07251worker2 # [ 235.052751] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:04Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17252worker2 # [ 235.092369] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418747253worker2 # [ 235.108340] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418747254client # copying 1 paths to 'lb:50051'...7255client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7256worker2 # [ 235.123473] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418747257worker2 # [ 235.140262] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7258worker2 # [ 235.309673] nix-daemon[859]: accepted connection from pid 2098, user nix-grpc-daemon (trusted)7259worker2 # [ 235.329818] nix-daemon[859]: reaped child process 2101, status = succeeded7260lb # [ 235.346404] niks3-server[1166]: 2026/09/23 04:47:04 INFO Received uploads request method=POST path=/api/pending_closures7261worker2 # [ 235.344315] nix-grpc-daemon[1801]: time=2026-09-23T04:47:04.911Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7262worker2 # [ 235.345940] nix-grpc-daemon[1801]: time=2026-09-23T04:47:04.911Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7263lb # [ 235.395676] niks3-server[1166]: 2026/09/23 04:47:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7264worker2 # [ 235.389363] nix-grpc-daemon[1801]: time=2026-09-23T04:47:04.957Z level=INFO msg="Uploading 1 narinfos"7265lb # [ 235.408537] niks3-server[1166]: 2026/09/23 04:47:04 INFO Signed narinfos id=88 count=17266lb # [ 235.430321] niks3-server[1166]: 2026/09/23 04:47:04 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7267lb # [ 235.439490] niks3-server[1166]: 2026/09/23 04:47:05 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7268lb # [ 235.450425] niks3-server[1166]: 2026/09/23 04:47:05 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7269lb # [ 235.459969] niks3-server[1166]: 2026/09/23 04:47:05 INFO Completed upload id=887270worker2 # [ 235.442843] nix-grpc-daemon[1801]: time=2026-09-23T04:47:05.010Z level=INFO msg="Upload complete. (285ms)"7271lb # [ 235.464883] niks3-server[1166]: 2026/09/23 04:47:05 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7272worker2 # [ 235.452370] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=5127273worker2 # [ 235.471875] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418747274worker2 # [ 235.483849] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7275worker2 # [ 235.487479] nix-daemon[859]: reaped child process 2096, status = succeeded7276client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7277client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7278worker2 # [ 235.503460] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7279client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7280client # farm-a> LOG-farm-a-drain7281worker2 # [ 236.997673] nix-daemon[859]: reaped child process 2105, status = succeeded7282worker2 # [ 237.081237] nix-daemon[859]: accepted connection from pid 2121, user nix-grpc-daemon (trusted)7283worker2 # [ 237.099756] nix-daemon[859]: reaped child process 2124, status = succeeded7284lb # [ 237.126562] niks3-server[1166]: 2026/09/23 04:47:06 INFO Received uploads request method=POST path=/api/pending_closures7285worker2 # [ 237.121803] nix-grpc-daemon[1801]: time=2026-09-23T04:47:06.687Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7286worker2 # [ 237.123594] nix-grpc-daemon[1801]: time=2026-09-23T04:47:06.687Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7287lb # [ 237.174763] niks3-server[1166]: 2026/09/23 04:47:06 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7288worker2 # [ 237.167170] nix-grpc-daemon[1801]: time=2026-09-23T04:47:06.734Z level=INFO msg="Uploading 1 narinfos"7289lb # [ 237.186907] niks3-server[1166]: 2026/09/23 04:47:06 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7290lb # [ 237.196294] niks3-server[1166]: 2026/09/23 04:47:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7291lb # [ 237.205793] niks3-server[1166]: 2026/09/23 04:47:06 INFO Signed narinfos id=89 count=17292lb # [ 237.214719] niks3-server[1166]: 2026/09/23 04:47:06 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7293worker2 # [ 237.197861] nix-grpc-daemon[1801]: time=2026-09-23T04:47:06.765Z level=INFO msg="Upload complete. (281ms)"7294lb # [ 237.224934] niks3-server[1166]: 2026/09/23 04:47:06 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7295lb # [ 237.231345] niks3-server[1166]: 2026/09/23 04:47:06 INFO Completed upload id=897296lb # [ 237.234412] niks3-server[1166]: 2026/09/23 04:47:06 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7297worker2 # [ 237.215643] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=17298worker2 # [ 237.236412] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:06Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07299worker2 # [ 237.242210] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:06Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17300worker2 # [ 237.247489] nix-daemon[859]: reaped child process 2103, status = succeeded7301worker2 # [ 237.263661] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418747302worker2 # [ 237.278839] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418747303worker2 # [ 237.293296] nix-grpc-daemon[1801]: time=2026-09-23T04:47:06.861Z level=INFO msg="All 1 paths already cached"7304client # copying 1 paths to 'lb:50051'...7305client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7306worker2 # [ 237.304418] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418747307worker2 # [ 237.320301] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7308worker2 # [ 237.471771] nix-daemon[859]: accepted connection from pid 2130, user nix-grpc-daemon (trusted)7309worker2 # [ 237.492391] nix-daemon[859]: reaped child process 2133, status = succeeded7310lb # [ 237.507506] niks3-server[1166]: 2026/09/23 04:47:07 INFO Received uploads request method=POST path=/api/pending_closures7311worker2 # [ 237.500887] nix-grpc-daemon[1801]: time=2026-09-23T04:47:07.068Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7312worker2 # [ 237.502724] nix-grpc-daemon[1801]: time=2026-09-23T04:47:07.068Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7313lb # [ 237.536379] niks3-server[1166]: 2026/09/23 04:47:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7314worker2 # [ 237.526169] nix-grpc-daemon[1801]: time=2026-09-23T04:47:07.094Z level=INFO msg="Uploading 1 narinfos"7315lb # [ 237.551806] niks3-server[1166]: 2026/09/23 04:47:07 INFO Signed narinfos id=90 count=17316lb # [ 237.557898] niks3-server[1166]: 2026/09/23 04:47:07 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7317worker2 # [ 237.553441] nix-grpc-daemon[1801]: time=2026-09-23T04:47:07.121Z level=INFO msg="Upload complete. (217ms)"7318lb # [ 237.570285] niks3-server[1166]: 2026/09/23 04:47:07 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7319worker2 # [ 237.560485] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=6327320lb # [ 237.577371] niks3-server[1166]: 2026/09/23 04:47:07 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7321lb # [ 237.586209] niks3-server[1166]: 2026/09/23 04:47:07 INFO Completed upload id=907322lb # [ 237.588812] niks3-server[1166]: 2026/09/23 04:47:07 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7323worker2 # [ 237.574798] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418747324worker2 # [ 237.580691] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7325worker2 # [ 237.585533] nix-daemon[859]: reaped child process 2128, status = succeeded7326client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7327worker2 # [ 237.597370] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7328client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7329client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7330lb # [ 238.668847] envoy[897]: conn peer=[2001:db8:1::1]:45542 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300777331client # farm-b> LOG-farm-b-drain7332worker2 # [ 238.869025] nix-daemon[859]: reaped child process 2137, status = succeeded7333worker2 # [ 238.981511] nix-daemon[859]: accepted connection from pid 2153, user nix-grpc-daemon (trusted)7334worker2 # [ 238.998141] nix-daemon[859]: reaped child process 2156, status = succeeded7335lb # [ 239.020750] niks3-server[1166]: 2026/09/23 04:47:08 INFO Received uploads request method=POST path=/api/pending_closures7336worker2 # [ 239.016439] nix-grpc-daemon[1801]: time=2026-09-23T04:47:08.581Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7337worker2 # [ 239.018273] nix-grpc-daemon[1801]: time=2026-09-23T04:47:08.581Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7338lb # [ 239.065681] niks3-server[1166]: 2026/09/23 04:47:08 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7339worker2 # [ 239.061701] nix-grpc-daemon[1801]: time=2026-09-23T04:47:08.629Z level=INFO msg="Uploading 1 narinfos"7340lb # [ 239.080575] niks3-server[1166]: 2026/09/23 04:47:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7341lb # [ 239.091371] niks3-server[1166]: 2026/09/23 04:47:08 INFO Signed narinfos id=91 count=17342lb # [ 239.096468] niks3-server[1166]: 2026/09/23 04:47:08 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7343lb # [ 239.099629] niks3-server[1166]: 2026/09/23 04:47:08 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7344lb # [ 239.110853] niks3-server[1166]: 2026/09/23 04:47:08 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7345worker2 # [ 239.093895] nix-grpc-daemon[1801]: time=2026-09-23T04:47:08.661Z level=INFO msg="Upload complete. (287ms)"7346lb # [ 239.113991] niks3-server[1166]: 2026/09/23 04:47:08 INFO Completed upload id=917347lb # [ 239.116660] niks3-server[1166]: 2026/09/23 04:47:08 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7348worker2 # [ 239.105883] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=17349worker2 # [ 239.123668] nix-daemon[859]: reaped child process 2135, status = succeeded7350worker2 # [ 239.156249] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:08Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07351worker2 # [ 239.162224] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:08Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17352worker2 # [ 239.172219] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418747353worker2 # [ 239.183507] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418747354worker2 # [ 239.197076] nix-grpc-daemon[1801]: time=2026-09-23T04:47:08.765Z level=INFO msg="All 2 paths already cached"7355client # copying 1 paths to 'lb:50051'...7356client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7357worker2 # [ 239.208144] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418747358worker2 # [ 239.221862] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7359worker2 # [ 239.368437] nix-daemon[859]: accepted connection from pid 2162, user nix-grpc-daemon (trusted)7360worker2 # [ 239.388411] nix-daemon[859]: reaped child process 2165, status = succeeded7361lb # [ 239.404174] niks3-server[1166]: 2026/09/23 04:47:08 INFO Received uploads request method=POST path=/api/pending_closures7362worker2 # [ 239.395637] nix-grpc-daemon[1801]: time=2026-09-23T04:47:08.963Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7363worker2 # [ 239.397336] nix-grpc-daemon[1801]: time=2026-09-23T04:47:08.963Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7364lb # [ 239.432569] niks3-server[1166]: 2026/09/23 04:47:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7365worker2 # [ 239.422196] nix-grpc-daemon[1801]: time=2026-09-23T04:47:08.990Z level=INFO msg="Uploading 1 narinfos"7366lb # [ 239.446614] niks3-server[1166]: 2026/09/23 04:47:09 INFO Signed narinfos id=92 count=17367lb # [ 239.454698] niks3-server[1166]: 2026/09/23 04:47:09 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7368lb # [ 239.468567] niks3-server[1166]: 2026/09/23 04:47:09 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7369worker2 # [ 239.458047] nix-grpc-daemon[1801]: time=2026-09-23T04:47:09.026Z level=INFO msg="Upload complete. (221ms)"7370lb # [ 239.478389] niks3-server[1166]: 2026/09/23 04:47:09 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7371lb # [ 239.484277] niks3-server[1166]: 2026/09/23 04:47:09 INFO Completed upload id=927372lb # [ 239.488648] niks3-server[1166]: 2026/09/23 04:47:09 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7373worker2 # [ 239.468360] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=0 paths=1 nar_bytes_in=7607374worker2 # [ 239.483949] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418747375worker2 # [ 239.493011] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7376worker2 # [ 239.496131] nix-daemon[859]: reaped child process 2160, status = succeeded7377client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7378worker2 # [ 239.508553] nix-daemon[859]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)7379client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7380client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7381client # farm-top> LOG-farm-top-drain7382worker2 # [ 240.607925] nix-daemon[859]: reaped child process 2169, status = succeeded7383worker2 # [ 240.675674] nix-daemon[859]: accepted connection from pid 2185, user nix-grpc-daemon (trusted)7384worker2 # [ 240.691755] nix-daemon[859]: reaped child process 2188, status = succeeded7385lb # [ 240.715141] niks3-server[1166]: 2026/09/23 04:47:10 INFO Received uploads request method=POST path=/api/pending_closures7386worker2 # [ 240.708375] nix-grpc-daemon[1801]: time=2026-09-23T04:47:10.275Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7387worker2 # [ 240.709962] nix-grpc-daemon[1801]: time=2026-09-23T04:47:10.275Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7388lb # [ 240.760557] niks3-server[1166]: 2026/09/23 04:47:10 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7389worker2 # [ 240.755288] nix-grpc-daemon[1801]: time=2026-09-23T04:47:10.323Z level=INFO msg="Uploading 1 narinfos"7390lb # [ 240.774020] niks3-server[1166]: 2026/09/23 04:47:10 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7391lb # [ 240.782778] niks3-server[1166]: 2026/09/23 04:47:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7392lb # [ 240.790777] niks3-server[1166]: 2026/09/23 04:47:10 INFO Signed narinfos id=93 count=17393lb # [ 240.794008] niks3-server[1166]: 2026/09/23 04:47:10 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7394lb # [ 240.803493] niks3-server[1166]: 2026/09/23 04:47:10 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7395worker2 # [ 240.786776] nix-grpc-daemon[1801]: time=2026-09-23T04:47:10.354Z level=INFO msg="Upload complete. (268ms)"7396lb # [ 240.807178] niks3-server[1166]: 2026/09/23 04:47:10 INFO Completed upload id=937397lb # [ 240.810586] niks3-server[1166]: 2026/09/23 04:47:10 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7398worker2 # [ 240.801833] nix-grpc-daemon[1724]: ts=2026-09-23T04:47:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41874 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=17399worker2 # [ 240.818954] nix-daemon[859]: reaped child process 2167, status = succeeded7400client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.07 seconds)7401client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7402client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7403worker1: must succeed: rm /nix/.rw-store/fill7404worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7405worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7406worker1 # [ 241.985286] nix-grpc-daemon[1616]: ts=2026-09-23T04:47:11Z level=info event=healthy reason=min_free7407worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7408(finished: subtest: low disk drains a worker and builds go to the other, in 9.92 seconds)7409(finished: run the VM test script, in 243.32 seconds)7410test script finished in 243.58s7411cleanup7412kill QemuMachine (pid 50)7413client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7414client # [2026-09-23T04:47:12Z INFO virtiofsd] Client disconnected, shutting down7415client # [2026-09-23T04:47:12Z INFO virtiofsd] Client disconnected, shutting down7416client # [2026-09-23T04:47:12Z INFO virtiofsd] Client disconnected, shutting down7417kill QemuMachine (pid 49)7418lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7419lb # [2026-09-23T04:47:12Z INFO virtiofsd] Client disconnected, shutting down7420lb # [2026-09-23T04:47:12Z INFO virtiofsd] Client disconnected, shutting down7421lb # [2026-09-23T04:47:12Z INFO virtiofsd] Client disconnected, shutting down7422kill QemuMachine (pid 51)7423worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7424worker1 # [2026-09-23T04:47:13Z INFO virtiofsd] Client disconnected, shutting down7425worker1 # [2026-09-23T04:47:13Z INFO virtiofsd] Client disconnected, shutting down7426worker1 # [2026-09-23T04:47:13Z INFO virtiofsd] Client disconnected, shutting down7427kill QemuMachine (pid 52)7428worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7429worker2 # [2026-09-23T04:47:13Z INFO virtiofsd] Client disconnected, shutting down7430worker2 # [2026-09-23T04:47:13Z INFO virtiofsd] Client disconnected, shutting down7431worker2 # [2026-09-23T04:47:13Z INFO virtiofsd] Client disconnected, shutting down7432(finished: cleanup, in 1.20 seconds)7433additionally exposed symbols:7434 client, lb, worker1, worker2,7435 vlan1,7436 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh