vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #548
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.A64hm4GJOD', fmt=raw size=107374182414worker2 # mke2fs 1.47.4 (6-Mar-2025)15worker2 # Discarding device blocks: 0/262144 done16worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes17lb # Disk image does not exist, creating the virtualisation disk image...18worker2 # Filesystem UUID: f9c82e2c-8814-4592-9a51-fe3bb167e28519lb # Formatting '/build/vm-state-lb/tmp.21k70Ixftg', fmt=raw size=107374182420worker2 # Superblock backups stored on blocks:21worker2: QEMU running (pid 52)22lb # mke2fs 1.47.4 (6-Mar-2025)23lb # Discarding device blocks: 0/262144 done24lb: QEMU running (pid 48)25lb # Creating filesystem with 262144 4k blocks and 65536 inodes26worker2 # 32768, 98304, 163840, 22937627lb # Filesystem UUID: e419f460-32a8-4ac9-96c7-3ad177ebeeda28client # Disk image does not exist, creating the virtualisation disk image...29client # Formatting '/build/vm-state-client/tmp.mvIEBejypW', fmt=raw size=107374182430worker1 # Disk image does not exist, creating the virtualisation disk image...31worker1: QEMU running (pid 51)32client # mke2fs 1.47.4 (6-Mar-2025)33client: QEMU running (pid 46)34worker1 # Formatting '/build/vm-state-worker1/tmp.J7aeQ8aePv', fmt=raw size=107374182435worker2 # 36worker2 # Allocating group tables: 0/8 done37lb # Superblock backups stored on blocks:38worker1 # mke2fs 1.47.4 (6-Mar-2025)39worker2 # Writing inode tables: 0/8 done40worker2 # Creating journal (8192 blocks): done41client # Discarding device blocks: 0/262144 done42worker2 # Writing superblocks and filesystem accounting information: 0/8 done43lb # 32768, 98304, 163840, 22937644client # Creating filesystem with 262144 4k blocks and 65536 inodes45lb # 46worker1 # Discarding device blocks: 0/262144 done47(finished: start all VMs, in 0.64 seconds)48worker2 # 49client # Filesystem UUID: 377aa17b-16db-4838-8736-3487cf584fcc50lb # Allocating group tables: 0/8 done51client: waiting for unit network-addresses-eth1.service52client: waiting for the VM to finish booting53worker2 # Virtualisation disk image created.54worker2 # Starting virtiofs daemons...55worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)56worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether57worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)58worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether59worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)60worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether61worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...62worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...63worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...64worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests65worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests66worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests67lb # Writing inode tables: 0/8 done68lb # Creating journal (8192 blocks): done69lb # Writing superblocks and filesystem accounting information: 0/8 done70lb # 71lb # Virtualisation disk image created.72lb # Starting virtiofs daemons...73lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether75lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79lb # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...80lb # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...81lb # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...82lb # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests83lb # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests84lb # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests85client # Superblock backups stored on blocks:86client # 32768, 98304, 163840, 22937687worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes88worker1 # Filesystem UUID: a7ecca0c-350f-44fc-aa57-2fc72629c49689worker1 # Superblock backups stored on blocks:90worker1 # 32768, 98304, 163840, 22937691worker1 # 92worker1 # Allocating group tables: 0/8 done93worker1 # Writing inode tables: 0/8 done94worker1 # Creating journal (8192 blocks): done95worker1 # Writing superblocks and filesystem accounting information: 0/8 done96worker1 # 97worker1 # Virtualisation disk image created.98worker1 # Starting virtiofs daemons...99worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...102worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)103worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...105worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)106worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether107worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...108worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests109worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests110worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests111client # 112client # Allocating group tables: 0/8 done113client # Writing inode tables: 0/8 done114client # Creating journal (8192 blocks): done115client # Writing superblocks and filesystem accounting information: 0/8 done116client # 117client # Virtualisation disk image created.118client # Starting virtiofs daemons...119client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)122client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123client # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...124client # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...125client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127client # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests131worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker2 # [ 0.000000] KASLR enabled134worker2 # [ 0.000000] random: crng init done135worker2 # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]137worker2 # [ 0.000000] efi: UEFI not found.138worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT139client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026140worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]141client # [ 0.000000] KASLR enabled142client # [ 0.000000] random: crng init done143worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]144worker2 # [ 0.000000] Zone ranges:145client # [ 0.000000] Machine model: linux,dummy-virt146client # [ 0.000000] efi: UEFI not found.147worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]148worker2 # [ 0.000000] DMA32 empty149worker2 # [ 0.000000] Normal empty150worker2 # [ 0.000000] Device empty151client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT152worker2 # [ 0.000000] Movable zone start for each node153client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]154worker2 # [ 0.000000] Early memory node ranges155client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]156client # [ 0.000000] Zone ranges:157worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]158client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]159client # [ 0.000000] DMA32 empty160worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]161worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000162worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]163worker2 # [ 0.000000] psci: probing for conduit method from DT.164worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.165worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs166worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026167client # [ 0.000000] Normal empty168worker1 # [ 0.000000] KASLR enabled169lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]170client # [ 0.000000] Device empty171client # [ 0.000000] Movable zone start for each node172client # [ 0.000000] Early memory node ranges173worker2 # [ 0.000000] psci: Trusted OS migration not required174client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]175client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]176worker1 # [ 0.000000] random: crng init done177lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026178worker2 # [ 0.000000] psci: SMC Calling Convention v1.1179client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000180worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)181lb # [ 0.000000] KASLR enabled182worker1 # [ 0.000000] Machine model: linux,dummy-virt183worker1 # [ 0.000000] efi: UEFI not found.184client # [ 0.000000] psci: probing for conduit method from DT.185client # [ 0.000000] psci: PSCIv1.3 detected in firmware.186worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT187lb # [ 0.000000] random: crng init done188client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs189worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296190lb # [ 0.000000] Machine model: linux,dummy-virt191client # [ 0.000000] psci: Trusted OS migration not required192lb # [ 0.000000] efi: UEFI not found.193worker2 # [ 0.000000] Detected PIPT I-cache on CPU0194worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]195worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]196client # [ 0.000000] psci: SMC Calling Convention v1.1197worker1 # [ 0.000000] Zone ranges:198lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT199worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)200client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)201worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface202worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]203lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]204worker1 # [ 0.000000] DMA32 empty205worker2 # [ 0.000000] CPU features: detected: Spectre-v4206client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296207worker1 # [ 0.000000] Normal empty208worker2 # [ 0.000000] CPU features: detected: Spectre-BHB209lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]210worker1 # [ 0.000000] Device empty211client # [ 0.000000] Detected PIPT I-cache on CPU0212lb # [ 0.000000] Zone ranges:213worker1 # [ 0.000000] Movable zone start for each node214worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38215client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)216lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]217worker1 # [ 0.000000] Early memory node ranges218worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23219lb # [ 0.000000] DMA32 empty220client # [ 0.000000] CPU features: detected: GICv3 CPU interface221worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]222lb # [ 0.000000] Normal empty223worker2 # [ 0.000000] alternatives: applying boot alternatives224lb # [ 0.000000] Device empty225client # [ 0.000000] CPU features: detected: Spectre-v4226worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]227lb # [ 0.000000] Movable zone start for each node228client # [ 0.000000] CPU features: detected: Spectre-BHB229lb # [ 0.000000] Early memory node ranges230worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000231client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38232worker1 # [ 0.000000] psci: probing for conduit method from DT.233lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]234client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23235worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.236worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs237client # [ 0.000000] alternatives: applying boot alternatives238lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]239lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000240worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6fvhs2ngm0npgxjv84bzdcnf0823nkly-nixos-system-worker2-test/init regInfo=/nix/store/i80rg3z8f160nwf6i6wkg9y7ng75s232-closure-info/registration console=ttyAMA0,115200n8 console=tty0241worker1 # [ 0.000000] psci: Trusted OS migration not required242lb # [ 0.000000] psci: probing for conduit method from DT.243worker1 # [ 0.000000] psci: SMC Calling Convention v1.1244lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.245worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)246lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs247worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i80rg3z8f160nwf6i6wkg9y7ng75s232-closure-info/registration", will be passed to user space.248lb # [ 0.000000] psci: Trusted OS migration not required249worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296250worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes251lb # [ 0.000000] psci: SMC Calling Convention v1.1252worker1 # [ 0.000000] Detected PIPT I-cache on CPU0253client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test/init regInfo=/nix/store/bhkqppcdz1hbk04v8bdn54wi7d6k8amk-closure-info/registration console=ttyAMA0,115200n8 console=tty0254worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)255lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)256worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)257worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)258worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface259client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bhkqppcdz1hbk04v8bdn54wi7d6k8amk-closure-info/registration", will be passed to user space.260lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296261lb # [ 0.000000] Detected PIPT I-cache on CPU0262worker1 # [ 0.000000] CPU features: detected: Spectre-v4263worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB264worker1 # [ 0.000000] CPU features: detected: Spectre-BHB265lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)266client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes267worker2 # [ 0.000000] software IO TLB: area num 1.268worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB269worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38270lb # [ 0.000000] CPU features: detected: GICv3 CPU interface271client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)272client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)273client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB274client # [ 0.000000] software IO TLB: area num 1.275worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23276client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB277worker1 # [ 0.000000] alternatives: applying boot alternatives278worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)279worker2 # [ 0.000000] Fallback order for Node 0: 0280worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216281worker2 # [ 0.000000] Policy zone: DMA282worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off283client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)284worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1285lb # [ 0.000000] CPU features: detected: Spectre-v4286lb # [ 0.000000] CPU features: detected: Spectre-BHB287lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38288lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23289lb # [ 0.000000] alternatives: applying boot alternatives290client # [ 0.000000] Fallback order for Node 0: 0291worker2 # [ 0.000000] allocated 3145728 bytes of page_ext292client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216293worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/70lv2j0cfdc8kw2f0kxgczfdih2jlj3k-nixos-system-worker1-test/init regInfo=/nix/store/ynqd41fnq28nbq3cfqld13z1wi4vm2cy-closure-info/registration console=ttyAMA0,115200n8 console=tty0294client # [ 0.000000] Policy zone: DMA295client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off296worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/ynqd41fnq28nbq3cfqld13z1wi4vm2cy-closure-info/registration", will be passed to user space.297client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1298lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test/init regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration console=ttyAMA0,115200n8 console=tty0299client # [ 0.000000] allocated 3145728 bytes of page_ext300worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes301worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages302worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups303worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.304worker2 # [ 0.000000] rcu: RCU event tracing is enabled.305worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.306worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.307worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)308worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.309worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.310worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)311worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.312worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB313worker1 # [ 0.000000] software IO TLB: area num 1.314lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration", will be passed to user space.315lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes316lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)317lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)318worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB319client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages320client # [ 0.000000] ftrace: allocated 294 pages with 4 groups321client # [ 0.000000] rcu: Hierarchical RCU implementation.322client # [ 0.000000] rcu: RCU event tracing is enabled.323client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.324client # [ 0.000000] Trampoline variant of Tasks RCU enabled.325client # [ 0.000000] Rude variant of Tasks RCU enabled.326client # [ 0.000000] Tracing variant of Tasks RCU enabled.327worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1328worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.329client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.330worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.331worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.332client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1333client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.334worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0335worker2 # [ 0.000000] GICv3: 256 SPIs implemented336lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB337client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.338lb # [ 0.000000] software IO TLB: area num 1.339lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB340worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented341worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)342worker1 # [ 0.000000] Fallback order for Node 0: 0343worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216344worker1 # [ 0.000000] Policy zone: DMA345worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off346worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1347worker1 # [ 0.000000] allocated 3145728 bytes of page_ext348lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)349lb # [ 0.000000] Fallback order for Node 0: 0350lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216351lb # [ 0.000000] Policy zone: DMA352worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq353lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off354worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages355client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.356worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI357worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups358lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1359worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0360client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0361worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.362client # [ 0.000000] GICv3: 256 SPIs implemented363lb # [ 0.000000] allocated 3145728 bytes of page_ext364worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000365worker1 # [ 0.000000] rcu: RCU event tracing is enabled.366worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]367lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages368client # [ 0.000000] GICv3: 0 Extended SPIs implemented369client # [ 0.000000] Root IRQ handler: gic_handle_irq370lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups371worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.372client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI373lb # [ 0.000000] rcu: Hierarchical RCU implementation.374worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.375worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)376client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0377worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.378lb # [ 0.000000] rcu: RCU event tracing is enabled.379worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.380client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000381worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)382lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.383client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]384worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000385worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.386lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.387lb # [ 0.000000] Rude variant of Tasks RCU enabled.388client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)389worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000390worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1391lb # [ 0.000000] Tracing variant of Tasks RCU enabled.392worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.393worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.394client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)395lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.396client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000397worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns398lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1399worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.400worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).401client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000402worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.403lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.404client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.405worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0406worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns407lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.408worker1 # [ 0.000000] GICv3: 256 SPIs implemented409worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented410client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns411worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq412client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).413lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.414worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns415worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI416lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0417worker2 # [ 0.000043] arm-pv: using stolen time PV418client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns419worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0420lb # [ 0.000000] GICv3: 256 SPIs implemented421lb # [ 0.000000] GICv3: 0 Extended SPIs implemented422lb # [ 0.000000] Root IRQ handler: gic_handle_irq423client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns424worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000425worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]426worker2 # [ 0.000616] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)427worker2 # [ 0.000842] Console: colour dummy device 80x25428client # [ 0.000035] arm-pv: using stolen time PV429lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI430worker2 # [ 0.000852] printk: legacy console [tty0] enabled431lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0432worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)433client # [ 0.000446] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)434lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000435worker2 # [ 0.001053] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)436lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]437worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)438worker2 # [ 0.001061] pid_max: default: 32768 minimum: 301439worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000440lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)441worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000442worker2 # [ 0.001144] LSM: initializing lsm=capability,landlock,yama,bpf,ima443worker2 # [ 0.001347] landlock: Up and running.444worker2 # [ 0.001350] Yama: becoming mindful.445lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)446worker2 # [ 0.001940] LSM support for eBPF active447client # [ 0.000657] Console: colour dummy device 80x25448client # [ 0.000665] printk: legacy console [tty0] enabled449client # [ 0.000864] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)450client # [ 0.000871] pid_max: default: 32768 minimum: 301451lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000452worker2 # [ 0.002166] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)453lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000454client # [ 0.000951] LSM: initializing lsm=capability,landlock,yama,bpf,ima455worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.456worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns457worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).458client # [ 0.001082] landlock: Up and running.459lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.460worker2 # [ 0.002203] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)461worker2 # [ 0.004346] rcu: Hierarchical SRCU implementation.462client # [ 0.001086] Yama: becoming mindful.463client # [ 0.001597] LSM support for eBPF active464worker2 # [ 0.004355] rcu: Max phase no-delay instances is 1000.465worker2 # [ 0.005759] fsl-mc MSI: its@8080000 domain created466lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns467client # [ 0.001776] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)468lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).469worker2 # [ 0.005864] EFI services will not be available.470worker2 # [ 0.005973] smp: Bringing up secondary CPUs ...471client # [ 0.001809] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)472lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns473client # [ 0.003952] rcu: Hierarchical SRCU implementation.474worker2 # [ 0.005984] smp: Brought up 1 node, 1 CPU475worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns476worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns477worker1 # [ 0.000046] arm-pv: using stolen time PV478worker1 # [ 0.000503] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)479worker1 # [ 0.000761] Console: colour dummy device 80x25480lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns481client # [ 0.003960] rcu: Max phase no-delay instances is 1000.482client # [ 0.005418] fsl-mc MSI: its@8080000 domain created483worker2 # [ 0.005987] SMP: Total of 1 processors activated.484worker2 # [ 0.005994] CPU: All CPU(s) started at EL1485worker2 # [ 0.006007] CPU features: detected: Branch Target Identification486worker1 # [ 0.000770] printk: legacy console [tty0] enabled487worker1 # [ 0.000980] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)488worker1 # [ 0.000987] pid_max: default: 32768 minimum: 301489lb # [ 0.000039] arm-pv: using stolen time PV490lb # [ 0.000498] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)491lb # [ 0.000780] Console: colour dummy device 80x25492lb # [ 0.000789] printk: legacy console [tty0] enabled493worker1 # [ 0.001071] LSM: initializing lsm=capability,landlock,yama,bpf,ima494worker1 # [ 0.001253] landlock: Up and running.495worker1 # [ 0.001256] Yama: becoming mindful.496lb # [ 0.001002] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)497worker1 # [ 0.001831] LSM support for eBPF active498lb # [ 0.001010] pid_max: default: 32768 minimum: 301499worker2 # [ 0.006014] CPU features: detected: ARMv8.4 Translation Table Level500worker2 # [ 0.006017] CPU features: detected: Instruction cache invalidation not required for I/D coherence501worker2 # [ 0.006021] CPU features: detected: Data cache clean to the PoU not required for I/D coherence502client # [ 0.005520] EFI services will not be available.503client # [ 0.005616] smp: Bringing up secondary CPUs ...504client # [ 0.005627] smp: Brought up 1 node, 1 CPU505client # [ 0.005631] SMP: Total of 1 processors activated.506client # [ 0.005637] CPU: All CPU(s) started at EL1507client # [ 0.005650] CPU features: detected: Branch Target Identification508client # [ 0.005655] CPU features: detected: ARMv8.4 Translation Table Level509client # [ 0.005659] CPU features: detected: Instruction cache invalidation not required for I/D coherence510lb # [ 0.001097] LSM: initializing lsm=capability,landlock,yama,bpf,ima511lb # [ 0.001267] landlock: Up and running.512worker1 # [ 0.001993] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)513worker2 # [ 0.006025] CPU features: detected: Common not Private translations514worker2 # [ 0.006029] CPU features: detected: CRC32 instructions515lb # [ 0.001270] Yama: becoming mindful.516worker1 # [ 0.002027] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)517worker2 # [ 0.006032] CPU features: detected: Data cache clean to Point of Deep Persistence518client # [ 0.005663] CPU features: detected: Data cache clean to the PoU not required for I/D coherence519lb # [ 0.001791] LSM support for eBPF active520worker2 # [ 0.006036] CPU features: detected: Data cache clean to Point of Persistence521worker1 # [ 0.004168] rcu: Hierarchical SRCU implementation.522client # [ 0.005667] CPU features: detected: Common not Private translations523worker2 # [ 0.006039] CPU features: detected: Data independent timing control (DIT)524worker1 # [ 0.004175] rcu: Max phase no-delay instances is 1000.525lb # [ 0.001959] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)526worker2 # [ 0.006042] CPU features: detected: E0PD527client # [ 0.005670] CPU features: detected: CRC32 instructions528worker2 # [ 0.006045] CPU features: detected: Enhanced Counter Virtualization529lb # [ 0.001994] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)530client # [ 0.005673] CPU features: detected: Data cache clean to Point of Deep Persistence531worker1 # [ 0.005541] fsl-mc MSI: its@8080000 domain created532lb # [ 0.004102] rcu: Hierarchical SRCU implementation.533worker2 # [ 0.006049] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)534worker1 # [ 0.005644] EFI services will not be available.535client # [ 0.005677] CPU features: detected: Data cache clean to Point of Persistence536lb # [ 0.004113] rcu: Max phase no-delay instances is 1000.537worker1 # [ 0.005741] smp: Bringing up secondary CPUs ...538worker2 # [ 0.006052] CPU features: detected: Enhanced Virtualization Traps539lb # [ 0.005583] fsl-mc MSI: its@8080000 domain created540client # [ 0.005680] CPU features: detected: Data independent timing control (DIT)541worker1 # [ 0.005753] smp: Brought up 1 node, 1 CPU542worker2 # [ 0.006056] CPU features: detected: Fine Grained Traps543lb # [ 0.005691] EFI services will not be available.544client # [ 0.005683] CPU features: detected: E0PD545worker1 # [ 0.005757] SMP: Total of 1 processors activated.546lb # [ 0.005845] smp: Bringing up secondary CPUs ...547worker1 # [ 0.005762] CPU: All CPU(s) started at EL1548worker2 # [ 0.006060] CPU features: detected: Generic authentication (architected QARMA5 algorithm)549client # [ 0.005686] CPU features: detected: Enhanced Counter Virtualization550lb # [ 0.005857] smp: Brought up 1 node, 1 CPU551worker1 # [ 0.005777] CPU features: detected: Branch Target Identification552worker2 # [ 0.006065] CPU features: detected: RCpc load-acquire (LDAPR)553client # [ 0.005689] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)554worker2 # [ 0.006069] CPU features: detected: LSE atomic instructions555client # [ 0.005693] CPU features: detected: Enhanced Virtualization Traps556worker1 # [ 0.005782] CPU features: detected: ARMv8.4 Translation Table Level557lb # [ 0.005860] SMP: Total of 1 processors activated.558lb # [ 0.005868] CPU: All CPU(s) started at EL1559worker2 # [ 0.006072] CPU features: detected: Privileged Access Never560lb # [ 0.005882] CPU features: detected: Branch Target Identification561client # [ 0.005696] CPU features: detected: Fine Grained Traps562worker1 # [ 0.005785] CPU features: detected: Instruction cache invalidation not required for I/D coherence563worker2 # [ 0.006075] CPU features: detected: PMUv3564lb # [ 0.005888] CPU features: detected: ARMv8.4 Translation Table Level565client # [ 0.005701] CPU features: detected: Generic authentication (architected QARMA5 algorithm)566worker2 # [ 0.006078] CPU features: detected: RAS Extension Support567worker1 # [ 0.005789] CPU features: detected: Data cache clean to the PoU not required for I/D coherence568client # [ 0.005706] CPU features: detected: RCpc load-acquire (LDAPR)569lb # [ 0.005892] CPU features: detected: Instruction cache invalidation not required for I/D coherence570worker2 # [ 0.006081] CPU features: detected: RASv1p1 Extension Support571worker1 # [ 0.005793] CPU features: detected: Common not Private translations572client # [ 0.005709] CPU features: detected: LSE atomic instructions573worker2 # [ 0.006084] CPU features: detected: Random Number Generator574worker1 # [ 0.005796] CPU features: detected: CRC32 instructions575lb # [ 0.005896] CPU features: detected: Data cache clean to the PoU not required for I/D coherence576client # [ 0.005712] CPU features: detected: Privileged Access Never577worker2 # [ 0.006087] CPU features: detected: Speculation barrier (SB)578lb # [ 0.005899] CPU features: detected: Common not Private translations579client # [ 0.005715] CPU features: detected: PMUv3580worker1 # [ 0.005799] CPU features: detected: Data cache clean to Point of Deep Persistence581worker2 # [ 0.006089] CPU features: detected: Stage-2 Force Write-Back582lb # [ 0.005903] CPU features: detected: CRC32 instructions583client # [ 0.005721] CPU features: detected: RAS Extension Support584worker1 # [ 0.005803] CPU features: detected: Data cache clean to Point of Persistence585worker2 # [ 0.006092] CPU features: detected: TLB range maintenance instructions586client # [ 0.005724] CPU features: detected: RASv1p1 Extension Support587lb # [ 0.005906] CPU features: detected: Data cache clean to Point of Deep Persistence588worker1 # [ 0.005807] CPU features: detected: Data independent timing control (DIT)589worker2 # [ 0.006097] CPU features: detected: Speculative Store Bypassing Safe (SSBS)590client # [ 0.005727] CPU features: detected: Random Number Generator591worker1 # [ 0.005810] CPU features: detected: E0PD592lb # [ 0.005909] CPU features: detected: Data cache clean to Point of Persistence593worker2 # [ 0.006169] alternatives: applying system-wide alternatives594client # [ 0.005730] CPU features: detected: Speculation barrier (SB)595worker1 # [ 0.005813] CPU features: detected: Enhanced Counter Virtualization596lb # [ 0.005913] CPU features: detected: Data independent timing control (DIT)597client # [ 0.005732] CPU features: detected: Stage-2 Force Write-Back598worker2 # [ 0.009892] CPU features: detected: BBM Level 2 without TLB conflict abort599lb # [ 0.005916] CPU features: detected: E0PD600worker1 # [ 0.005816] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)601client # [ 0.005735] CPU features: detected: TLB range maintenance instructions602lb # [ 0.005919] CPU features: detected: Enhanced Counter Virtualization603worker1 # [ 0.005820] CPU features: detected: Enhanced Virtualization Traps604client # [ 0.005741] CPU features: detected: Speculative Store Bypassing Safe (SSBS)605worker2 # [ 0.010145] Memory: 1405692K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)606worker1 # [ 0.005823] CPU features: detected: Fine Grained Traps607lb # [ 0.005922] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)608worker2 # [ 0.010633] devtmpfs: initialized609client # [ 0.005784] alternatives: applying system-wide alternatives610lb # [ 0.005926] CPU features: detected: Enhanced Virtualization Traps611worker1 # [ 0.005827] CPU features: detected: Generic authentication (architected QARMA5 algorithm)612worker2 # [ 0.012728] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)613client # [ 0.009520] CPU features: detected: BBM Level 2 without TLB conflict abort614lb # [ 0.005929] CPU features: detected: Fine Grained Traps615worker1 # [ 0.005833] CPU features: detected: RCpc load-acquire (LDAPR)616worker2 # [ 0.012757] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).617worker1 # [ 0.005836] CPU features: detected: LSE atomic instructions618lb # [ 0.005933] CPU features: detected: Generic authentication (architected QARMA5 algorithm)619worker2 # [ 0.012989] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL620client # [ 0.009773] Memory: 1405644K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)621worker1 # [ 0.005839] CPU features: detected: Privileged Access Never622lb # [ 0.005938] CPU features: detected: RCpc load-acquire (LDAPR)623client # [ 0.010286] devtmpfs: initialized624worker2 # [ 0.012995] 0 pages in range for non-PLT usage625worker1 # [ 0.005842] CPU features: detected: PMUv3626lb # [ 0.005940] CPU features: detected: LSE atomic instructions627worker2 # [ 0.012997] 508272 pages in range for PLT usage628worker1 # [ 0.005845] CPU features: detected: RAS Extension Support629client # [ 0.012308] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)630lb # [ 0.005943] CPU features: detected: Privileged Access Never631worker2 # [ 0.013156] pinctrl core: initialized pinctrl subsystem632worker2 # [ 0.014193] DMI not present or invalid.633lb # [ 0.005946] CPU features: detected: PMUv3634lb # [ 0.005949] CPU features: detected: RAS Extension Support635lb # [ 0.005953] CPU features: detected: RASv1p1 Extension Support636worker2 # [ 0.017607] NET: Registered PF_NETLINK/PF_ROUTE protocol family637worker2 # [ 0.020298] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations638client # [ 0.012334] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).639worker1 # [ 0.005848] CPU features: detected: RASv1p1 Extension Support640client # [ 0.012572] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL641worker1 # [ 0.005851] CPU features: detected: Random Number Generator642client # [ 0.012577] 0 pages in range for non-PLT usage643lb # [ 0.005956] CPU features: detected: Random Number Generator644worker1 # [ 0.005853] CPU features: detected: Speculation barrier (SB)645client # [ 0.012579] 508272 pages in range for PLT usage646worker1 # [ 0.005856] CPU features: detected: Stage-2 Force Write-Back647worker2 # [ 0.020528] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations648client # [ 0.012723] pinctrl core: initialized pinctrl subsystem649lb # [ 0.005958] CPU features: detected: Speculation barrier (SB)650client # [ 0.013635] DMI not present or invalid.651client # [ 0.017123] NET: Registered PF_NETLINK/PF_ROUTE protocol family652worker1 # [ 0.005859] CPU features: detected: TLB range maintenance instructions653worker1 # [ 0.005865] CPU features: detected: Speculative Store Bypassing Safe (SSBS)654worker1 # [ 0.005911] alternatives: applying system-wide alternatives655worker2 # [ 0.020770] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations656worker2 # [ 0.020813] audit: initializing netlink subsys (disabled)657lb # [ 0.005961] CPU features: detected: Stage-2 Force Write-Back658lb # [ 0.005965] CPU features: detected: TLB range maintenance instructions659worker1 # [ 0.009834] CPU features: detected: BBM Level 2 without TLB conflict abort660worker2 # [ 0.021498] thermal_sys: Registered thermal governor 'fair_share'661client # [ 0.020097] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations662lb # [ 0.005970] CPU features: detected: Speculative Store Bypassing Safe (SSBS)663worker2 # [ 0.021501] thermal_sys: Registered thermal governor 'bang_bang'664client # [ 0.020315] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations665lb # [ 0.006016] alternatives: applying system-wide alternatives666worker2 # [ 0.021506] thermal_sys: Registered thermal governor 'step_wise'667lb # [ 0.009785] CPU features: detected: BBM Level 2 without TLB conflict abort668client # [ 0.020540] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations669worker1 # [ 0.010117] Memory: 1405600K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)670worker1 # [ 0.010647] devtmpfs: initialized671worker2 # [ 0.021509] thermal_sys: Registered thermal governor 'user_space'672client # [ 0.020574] audit: initializing netlink subsys (disabled)673worker1 # [ 0.012657] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)674client # [ 0.021244] thermal_sys: Registered thermal governor 'fair_share'675worker2 # [ 0.021515] thermal_sys: Registered thermal governor 'power_allocator'676client # [ 0.021247] thermal_sys: Registered thermal governor 'bang_bang'677worker2 # [ 0.021541] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1678worker1 # [ 0.012685] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).679worker2 # [ 0.021554] cpuidle: using governor ladder680client # [ 0.021251] thermal_sys: Registered thermal governor 'step_wise'681worker1 # [ 0.012916] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL682worker2 # [ 0.021560] cpuidle: using governor menu683client # [ 0.021254] thermal_sys: Registered thermal governor 'user_space'684worker1 # [ 0.012920] 0 pages in range for non-PLT usage685worker2 # [ 0.021776] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.686worker1 # [ 0.012922] 508272 pages in range for PLT usage687client # [ 0.021260] thermal_sys: Registered thermal governor 'power_allocator'688worker2 # [ 0.021798] ASID allocator initialised with 65536 entries689worker1 # [ 0.013061] pinctrl core: initialized pinctrl subsystem690lb # [ 0.010017] Memory: 1405636K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)691worker2 # [ 0.023162] Serial: AMBA PL011 UART driver692worker1 # [ 0.013962] DMI not present or invalid.693lb # [ 0.010515] devtmpfs: initialized694client # [ 0.021289] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1695lb # [ 0.012444] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)696lb # [ 0.012471] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).697lb # [ 0.012767] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL698client # [ 0.021301] cpuidle: using governor ladder699worker1 # [ 0.018178] NET: Registered PF_NETLINK/PF_ROUTE protocol family700lb # [ 0.012772] 0 pages in range for non-PLT usage701worker2 # [ 0.028633] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1702client # [ 0.021306] cpuidle: using governor menu703lb # [ 0.012773] 508272 pages in range for PLT usage704worker2 # [ 0.028810] printk: console [ttyAMA0] enabled705worker1 # [ 0.020827] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations706client # [ 0.021515] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.707lb # [ 0.012915] pinctrl core: initialized pinctrl subsystem708client # [ 0.021533] ASID allocator initialised with 65536 entries709lb # [ 0.013742] DMI not present or invalid.710client # [ 0.022871] Serial: AMBA PL011 UART driver711worker1 # [ 0.021063] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations712lb # [ 0.017361] NET: Registered PF_NETLINK/PF_ROUTE protocol family713worker1 # [ 0.021294] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations714client # [ 0.028448] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1715lb # [ 0.020145] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations716worker1 # [ 0.021334] audit: initializing netlink subsys (disabled)717client # [ 0.028678] printk: console [ttyAMA0] enabled718lb # [ 0.020382] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations719worker1 # [ 0.021889] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1720worker1 # [ 0.022082] thermal_sys: Registered thermal governor 'fair_share'721lb # [ 0.020644] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations722lb # [ 0.020687] audit: initializing netlink subsys (disabled)723worker1 # [ 0.022085] thermal_sys: Registered thermal governor 'bang_bang'724worker1 # [ 0.022089] thermal_sys: Registered thermal governor 'step_wise'725lb # [ 0.021388] thermal_sys: Registered thermal governor 'fair_share'726worker1 # [ 0.022095] thermal_sys: Registered thermal governor 'user_space'727lb # [ 0.021390] thermal_sys: Registered thermal governor 'bang_bang'728worker1 # [ 0.022097] thermal_sys: Registered thermal governor 'power_allocator'729lb # [ 0.021394] thermal_sys: Registered thermal governor 'step_wise'730worker2 # [ 0.154875] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages731worker1 # [ 0.022125] cpuidle: using governor ladder732lb # [ 0.021397] thermal_sys: Registered thermal governor 'user_space'733worker1 # [ 0.022133] cpuidle: using governor menu734worker2 # [ 0.154902] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page735lb # [ 0.021403] thermal_sys: Registered thermal governor 'power_allocator'736worker1 # [ 0.022349] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.737worker2 # [ 0.154908] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages738worker1 # [ 0.022368] ASID allocator initialised with 65536 entries739lb # [ 0.021435] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1740worker2 # [ 0.154913] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page741worker1 # [ 0.023682] Serial: AMBA PL011 UART driver742lb # [ 0.021444] cpuidle: using governor ladder743worker2 # [ 0.154918] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages744lb # [ 0.021450] cpuidle: using governor menu745worker1 # [ 0.029317] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1746worker2 # [ 0.154922] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page747lb # [ 0.021666] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.748worker1 # [ 0.029528] printk: console [ttyAMA0] enabled749client # [ 0.156651] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages750worker2 # [ 0.154927] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages751lb # [ 0.021683] ASID allocator initialised with 65536 entries752lb # [ 0.023083] Serial: AMBA PL011 UART driver753worker2 # [ 0.154932] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page754lb # [ 0.028836] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1755lb # [ 0.029008] printk: console [ttyAMA0] enabled756client # [ 0.156680] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page757client # [ 0.156686] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages758client # [ 0.156690] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page759client # [ 0.156695] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages760worker2 # [ 0.163412] fbcon: Taking over console761client # [ 0.156700] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page762worker2 # [ 0.163435] ACPI: Interpreter disabled.763client # [ 0.156705] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages764client # [ 0.156709] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page765client # [ 0.164711] fbcon: Taking over console766client # [ 0.164733] ACPI: Interpreter disabled.767worker1 # [ 0.158384] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages768client # [ 0.166716] iommu: Default domain type: Translated769worker1 # [ 0.158412] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page770worker2 # [ 0.170761] iommu: Default domain type: Translated771lb # [ 0.156825] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages772client # [ 0.166730] iommu: DMA domain TLB invalidation policy: strict mode773worker1 # [ 0.158418] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages774worker2 # [ 0.170777] iommu: DMA domain TLB invalidation policy: strict mode775lb # [ 0.156853] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page776worker2 # [ 0.172788] SCSI subsystem initialized777worker1 # [ 0.158423] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page778lb # [ 0.156859] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages779worker1 # [ 0.158428] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages780lb # [ 0.156863] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page781worker1 # [ 0.158432] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page782worker2 # [ 0.173550] usbcore: registered new interface driver usbfs783lb # [ 0.156868] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages784worker1 # [ 0.158436] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages785worker2 # [ 0.173590] usbcore: registered new interface driver hub786lb # [ 0.156872] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page787worker2 # [ 0.173614] usbcore: registered new device driver usb788worker1 # [ 0.158440] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page789lb # [ 0.156876] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages790client # [ 0.172695] SCSI subsystem initialized791worker2 # [ 0.173933] pps_core: LinuxPPS API ver. 1 registered792lb # [ 0.156881] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page793worker2 # [ 0.173941] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>794client # [ 0.173476] usbcore: registered new interface driver usbfs795worker2 # [ 0.173951] PTP clock support registered796worker2 # [ 0.174006] EDAC MC: Ver: 3.0.0797client # [ 0.173513] usbcore: registered new interface driver hub798worker1 # [ 0.166676] fbcon: Taking over console799client # [ 0.173530] usbcore: registered new device driver usb800worker1 # [ 0.166697] ACPI: Interpreter disabled.801worker2 # [ 0.179043] scmi_core: SCMI protocol bus registered802client # [ 0.173854] pps_core: LinuxPPS API ver. 1 registered803lb # [ 0.165226] fbcon: Taking over console804lb # [ 0.165249] ACPI: Interpreter disabled.805worker2 # [ 0.180140] FPGA manager framework806client # [ 0.173862] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>807client # [ 0.173873] PTP clock support registered808worker2 # [ 0.181272] vgaarb: loaded809client # [ 0.173938] EDAC MC: Ver: 3.0.0810worker2 # [ 0.181983] clocksource: Switched to clocksource arch_sys_counter811client # [ 0.178851] scmi_core: SCMI protocol bus registered812client # [ 0.179897] FPGA manager framework813client # [ 0.181011] vgaarb: loaded814worker1 # [ 0.174108] iommu: Default domain type: Translated815worker1 # [ 0.174122] iommu: DMA domain TLB invalidation policy: strict mode816lb # [ 0.172538] iommu: Default domain type: Translated817worker1 # [ 0.176039] SCSI subsystem initialized818lb # [ 0.172554] iommu: DMA domain TLB invalidation policy: strict mode819lb # [ 0.173043] SCSI subsystem initialized820worker2 # [ 0.188260] VFS: Disk quotas dquot_6.6.0821worker1 # [ 0.176825] usbcore: registered new interface driver usbfs822worker1 # [ 0.176864] usbcore: registered new interface driver hub823worker2 # [ 0.188317] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)824client # [ 0.185225] clocksource: Switched to clocksource arch_sys_counter825lb # [ 0.175298] usbcore: registered new interface driver usbfs826worker1 # [ 0.176880] usbcore: registered new device driver usb827client # [ 0.185969] VFS: Disk quotas dquot_6.6.0828worker1 # [ 0.177230] pps_core: LinuxPPS API ver. 1 registered829lb # [ 0.175336] usbcore: registered new interface driver hub830lb # [ 0.175352] usbcore: registered new device driver usb831worker1 # [ 0.177237] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>832client # [ 0.186013] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)833lb # [ 0.175778] pps_core: LinuxPPS API ver. 1 registered834client # [ 0.188541] netfs: FS-Cache loaded835client # [ 0.188696] pnp: PnP ACPI: disabled836worker1 # [ 0.177248] PTP clock support registered837worker1 # [ 0.177299] EDAC MC: Ver: 3.0.0838worker2 # [ 0.192682] netfs: FS-Cache loaded839lb # [ 0.175791] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>840worker2 # [ 0.192870] pnp: PnP ACPI: disabled841worker1 # [ 0.182442] scmi_core: SCMI protocol bus registered842lb # [ 0.175803] PTP clock support registered843lb # [ 0.175864] EDAC MC: Ver: 3.0.0844worker1 # [ 0.183591] FPGA manager framework845lb # [ 0.181449] scmi_core: SCMI protocol bus registered846worker1 # [ 0.184752] vgaarb: loaded847worker1 # [ 0.185474] clocksource: Switched to clocksource arch_sys_counter848lb # [ 0.182660] FPGA manager framework849lb # [ 0.183868] vgaarb: loaded850worker2 # [ 0.197308] NET: Registered PF_INET protocol family851lb # [ 0.184668] clocksource: Switched to clocksource arch_sys_counter852worker2 # [ 0.197639] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)853client # [ 0.195555] NET: Registered PF_INET protocol family854client # [ 0.195912] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)855worker1 # [ 0.192129] VFS: Disk quotas dquot_6.6.0856worker1 # [ 0.192216] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)857lb # [ 0.191225] VFS: Disk quotas dquot_6.6.0858worker1 # [ 0.194215] netfs: FS-Cache loaded859worker1 # [ 0.194375] pnp: PnP ACPI: disabled860lb # [ 0.191296] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)861lb # [ 0.195667] netfs: FS-Cache loaded862lb # [ 0.195852] pnp: PnP ACPI: disabled863worker1 # [ 0.198721] NET: Registered PF_INET protocol family864worker1 # [ 0.199040] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)865lb # [ 0.200699] NET: Registered PF_INET protocol family866lb # [ 0.201066] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)867client # [ 0.231611] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)868client # [ 0.231687] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)869client # [ 0.231728] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)870client # [ 0.231838] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)871client # [ 0.231985] TCP: Hash tables configured (established 16384 bind 16384)872client # [ 0.232106] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)873client # [ 0.232153] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)874worker2 # [ 0.232843] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)875client # [ 0.232194] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)876worker2 # [ 0.232918] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)877client # [ 0.232313] NET: Registered PF_UNIX/PF_LOCAL protocol family878worker2 # [ 0.232959] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)879client # [ 0.232344] NET: Registered PF_XDP protocol family880client # [ 0.232367] PCI: CLS 0 bytes, default 64881worker2 # [ 0.233060] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)882worker2 # [ 0.233206] TCP: Hash tables configured (established 16384 bind 16384)883worker2 # [ 0.233325] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)884worker2 # [ 0.233383] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)885worker2 # [ 0.233425] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)886worker2 # [ 0.233534] NET: Registered PF_UNIX/PF_LOCAL protocol family887worker2 # [ 0.233558] NET: Registered PF_XDP protocol family888worker2 # [ 0.233579] PCI: CLS 0 bytes, default 64889worker2 # [ 0.233882] Trying to unpack rootfs image as initramfs...890client # [ 0.242800] Trying to unpack rootfs image as initramfs...891worker1 # [ 0.234690] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)892worker1 # [ 0.234764] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)893worker1 # [ 0.234802] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)894worker1 # [ 0.234913] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)895worker1 # [ 0.235059] TCP: Hash tables configured (established 16384 bind 16384)896worker1 # [ 0.235223] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)897worker2 # [ 0.252667] kvm [1]: HYP mode not available898worker1 # [ 0.235276] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)899worker1 # [ 0.235318] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker1 # [ 0.235435] NET: Registered PF_UNIX/PF_LOCAL protocol family901worker1 # [ 0.235501] NET: Registered PF_XDP protocol family902client # [ 0.250157] kvm [1]: HYP mode not available903worker1 # [ 0.235522] PCI: CLS 0 bytes, default 64904worker1 # [ 0.235813] Trying to unpack rootfs image as initramfs...905lb # [ 0.239829] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)906lb # [ 0.239902] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)907worker1 # [ 0.251414] kvm [1]: HYP mode not available908lb # [ 0.239943] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909lb # [ 0.240053] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910lb # [ 0.240202] TCP: Hash tables configured (established 16384 bind 16384)911lb # [ 0.240326] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912lb # [ 0.240377] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913lb # [ 0.240418] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914lb # [ 0.240550] NET: Registered PF_UNIX/PF_LOCAL protocol family915lb # [ 0.240625] NET: Registered PF_XDP protocol family916lb # [ 0.240649] PCI: CLS 0 bytes, default 64917lb # [ 0.240785] Trying to unpack rootfs image as initramfs...918lb # [ 0.259222] kvm [1]: HYP mode not available919worker2 # [ 0.463428] Initialise system trusted keyrings920worker2 # [ 0.464318] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker2 # [ 0.465758] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker2 # [ 0.466846] 9p: Installing v9fs 9p2000 file system support923client # [ 0.469939] Initialise system trusted keyrings924client # [ 0.470896] workingset: timestamp_bits=42 max_order=19 bucket_order=0925client # [ 0.472361] squashfs: version 4.0 (2009/01/31) Phillip Lougher926client # [ 0.473320] 9p: Installing v9fs 9p2000 file system support927worker1 # [ 0.471777] Initialise system trusted keyrings928worker1 # [ 0.472707] workingset: timestamp_bits=42 max_order=19 bucket_order=0929worker1 # [ 0.474366] squashfs: version 4.0 (2009/01/31) Phillip Lougher930worker1 # [ 0.475295] 9p: Installing v9fs 9p2000 file system support931lb # [ 0.481496] Initialise system trusted keyrings932worker2 # [ 0.495099] Key type asymmetric registered933worker2 # [ 0.495137] Asymmetric key parser 'x509' registered934lb # [ 0.482364] workingset: timestamp_bits=42 max_order=19 bucket_order=0935worker2 # [ 0.495235] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)936lb # [ 0.483817] squashfs: version 4.0 (2009/01/31) Phillip Lougher937worker2 # [ 0.497658] io scheduler mq-deadline registered938worker2 # [ 0.497672] io scheduler kyber registered939lb # [ 0.484810] 9p: Installing v9fs 9p2000 file system support940client # [ 0.501694] Key type asymmetric registered941client # [ 0.501729] Asymmetric key parser 'x509' registered942worker1 # [ 0.495510] Key type asymmetric registered943client # [ 0.501824] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)944worker1 # [ 0.495544] Asymmetric key parser 'x509' registered945client # [ 0.504370] io scheduler mq-deadline registered946client # [ 0.504386] io scheduler kyber registered947worker1 # [ 0.495642] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)948worker2 # [ 0.511632] pl061_gpio 9030000.pl061: PL061 GPIO chip registered949worker2 # [ 0.513112] ledtrig-cpu: registered to indicate activity on CPUs950worker2 # [ 0.513562] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:951worker2 # [ 0.513585] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000952worker1 # [ 0.509594] io scheduler mq-deadline registered953worker1 # [ 0.509628] io scheduler kyber registered954worker2 # [ 0.513597] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000955worker2 # [ 0.513606] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000956worker2 # [ 0.513638] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits957worker2 # [ 0.513665] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]958worker2 # [ 0.513764] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00959worker2 # [ 0.513774] pci_bus 0000:00: root bus resource [bus 00-ff]960lb # [ 0.513065] Key type asymmetric registered961worker1 # [ 0.515352] pl061_gpio 9030000.pl061: PL061 GPIO chip registered962lb # [ 0.513102] Asymmetric key parser 'x509' registered963worker2 # [ 0.513781] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]964worker2 # [ 0.513787] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]965lb # [ 0.513193] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)966worker1 # [ 0.516748] ledtrig-cpu: registered to indicate activity on CPUs967client # [ 0.518396] pl061_gpio 9030000.pl061: PL061 GPIO chip registered968worker2 # [ 0.513792] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]969worker1 # [ 0.517195] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:970client # [ 0.519184] ledtrig-cpu: registered to indicate activity on CPUs971lb # [ 0.515583] io scheduler mq-deadline registered972lb # [ 0.515598] io scheduler kyber registered973worker2 # [ 0.513856] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint974client # [ 0.519643] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:975worker1 # [ 0.517214] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000976worker2 # [ 0.514416] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint977client # [ 0.519665] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000978worker2 # [ 0.514614] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]979worker1 # [ 0.517227] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000980client # [ 0.519678] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000981worker1 # [ 0.517235] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000982worker2 # [ 0.514633] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]983worker2 # [ 0.514662] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]984client # [ 0.519687] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000985worker2 # [ 0.514678] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]986worker1 # [ 0.517270] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits987client # [ 0.519712] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits988worker2 # [ 0.515157] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint989worker1 # [ 0.517294] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]990worker2 # [ 0.515357] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]991worker1 # [ 0.517403] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00992client # [ 0.519736] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]993worker2 # [ 0.515373] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]994client # [ 0.519821] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00995worker2 # [ 0.515404] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996client # [ 0.519832] pci_bus 0000:00: root bus resource [bus 00-ff]997worker1 # [ 0.517413] pci_bus 0000:00: root bus resource [bus 00-ff]998worker1 # [ 0.517420] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]999worker1 # [ 0.517425] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1000worker2 # [ 0.515917] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1001client # [ 0.519838] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1002worker2 # [ 0.516121] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1003worker1 # [ 0.517430] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1004client # [ 0.519844] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1005worker2 # [ 0.516139] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1006client # [ 0.519849] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1007worker2 # [ 0.516171] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1008worker1 # [ 0.529023] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1009client # [ 0.519934] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1010worker2 # [ 0.516675] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1011worker2 # [ 0.516894] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1012client # [ 0.520442] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1013worker2 # [ 0.516913] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1014client # [ 0.520654] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1015client # [ 0.520675] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1016worker2 # [ 0.516947] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017worker2 # [ 0.516968] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1018client # [ 0.520707] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019client # [ 0.520726] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1020worker2 # [ 0.517469] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1021client # [ 0.521259] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1022worker2 # [ 0.517681] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1023client # [ 0.521470] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1024worker2 # [ 0.517716] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025client # [ 0.521487] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1026client # [ 0.521520] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1027lb # [ 0.529575] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1028client # [ 0.522040] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1029lb # [ 0.530211] ledtrig-cpu: registered to indicate activity on CPUs1030client # [ 0.522242] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1031client # [ 0.522260] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1032lb # [ 0.530633] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1033client # [ 0.522302] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1034lb # [ 0.530654] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001035worker1 # [ 0.537662] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1036worker1 # [ 0.537911] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1037client # [ 0.522846] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1038worker1 # [ 0.537934] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1039lb # [ 0.530668] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001040client # [ 0.523053] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1041worker1 # [ 0.537968] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042client # [ 0.523070] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1043lb # [ 0.530677] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001044worker1 # [ 0.537990] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1045client # [ 0.523101] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046lb # [ 0.530702] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1047client # [ 0.523123] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1048worker1 # [ 0.538603] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1049worker1 # [ 0.538818] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1050client # [ 0.523638] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1051worker1 # [ 0.538835] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1052client # [ 0.523844] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1053lb # [ 0.530727] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1054lb # [ 0.530817] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001055worker1 # [ 0.538866] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1056client # [ 0.523878] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057lb # [ 0.530827] pci_bus 0000:00: root bus resource [bus 00-ff]1058lb # [ 0.530834] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1059worker1 # [ 0.539426] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1060client # [ 0.524380] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1061lb # [ 0.530839] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1062worker1 # [ 0.539626] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1063client # [ 0.524586] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1064lb # [ 0.530844] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1065worker1 # [ 0.539643] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1066client # [ 0.524619] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067worker2 # [ 0.554481] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1068worker1 # [ 0.539677] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069lb # [ 0.530921] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1070worker2 # [ 0.554725] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1071client # [ 0.525042] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1072worker1 # [ 0.540181] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1073lb # [ 0.531415] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1074worker2 # [ 0.554763] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1075lb # [ 0.531619] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1076worker1 # [ 0.540384] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1077worker2 # [ 0.555215] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1078worker1 # [ 0.540403] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1079lb # [ 0.531639] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1080worker2 # [ 0.555415] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1081worker1 # [ 0.540438] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1082lb # [ 0.531671] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1083lb # [ 0.531689] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1084worker1 # [ 0.540459] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1085worker2 # [ 0.555711] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1086worker2 # [ 0.555916] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1087lb # [ 0.532178] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1088worker1 # [ 0.540981] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1089lb # [ 0.532380] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1090worker1 # [ 0.541187] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1091worker2 # [ 0.555950] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1092lb # [ 0.532398] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1093worker1 # [ 0.541222] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094worker2 # [ 0.556437] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1095lb # [ 0.532429] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1096worker2 # [ 0.556651] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1097lb # [ 0.532988] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1098worker2 # [ 0.556681] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1099client # [ 0.565354] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1100lb # [ 0.533174] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1101worker2 # [ 0.557272] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1102lb # [ 0.533190] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1103client # [ 0.565781] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1104worker2 # [ 0.557478] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1105lb # [ 0.533221] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1106client # [ 0.566015] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1107worker2 # [ 0.557510] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108client # [ 0.566050] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109worker2 # [ 0.558070] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1110lb # [ 0.533694] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1111worker1 # [ 0.560692] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1112lb # [ 0.533896] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1113client # [ 0.566584] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1114lb # [ 0.533912] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1115worker2 # [ 0.558409] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1116client # [ 0.566844] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1117worker1 # [ 0.560909] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1118lb # [ 0.533941] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1119worker2 # [ 0.558430] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1120worker1 # [ 0.560942] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1121client # [ 0.566880] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122lb # [ 0.533960] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1123worker2 # [ 0.558464] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124worker1 # [ 0.561366] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1125client # [ 0.567400] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1126lb # [ 0.534447] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1127worker2 # [ 0.558979] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1128worker1 # [ 0.561578] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1129client # [ 0.567611] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1130lb # [ 0.534641] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1131worker2 # [ 0.559194] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1132client # [ 0.567646] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133worker1 # [ 0.561884] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1134lb # [ 0.534672] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135worker2 # [ 0.559213] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1136worker1 # [ 0.562088] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1137client # [ 0.568183] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1138worker2 # [ 0.559247] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1139lb # [ 0.535152] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1140worker1 # [ 0.562119] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1141client # [ 0.568529] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1142lb # [ 0.535352] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker2 # [ 0.559890] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1144client # [ 0.568552] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1145worker1 # [ 0.562596] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1146lb # [ 0.535387] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147worker2 # [ 0.559903] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1148worker1 # [ 0.562796] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1149client # [ 0.568584] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1150lb # [ 0.535841] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1151worker2 # [ 0.559908] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1152worker1 # [ 0.562829] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153client # [ 0.569104] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1154lb # [ 0.536038] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1155worker2 # [ 0.559959] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1156worker1 # [ 0.563339] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1157lb # [ 0.536299] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1158worker1 # [ 0.563551] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1159worker2 # [ 0.560011] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1160lb # [ 0.536496] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1161worker1 # [ 0.563585] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1162worker2 # [ 0.560071] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1163lb # [ 0.536528] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164worker1 # [ 0.564080] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1165worker2 # [ 0.560120] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1166worker1 # [ 0.564425] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1167worker2 # [ 0.560176] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1168worker1 # [ 0.564447] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1169worker2 # [ 0.560228] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1170worker1 # [ 0.564483] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171worker2 # [ 0.560278] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1172worker1 # [ 0.565013] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1173worker1 # [ 0.565220] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1174worker2 # [ 0.560332] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1175worker1 # [ 0.565239] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1176worker2 # [ 0.560382] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1177worker1 # [ 0.565275] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1178worker2 # [ 0.560464] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1179worker2 # [ 0.560553] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1180client # [ 0.589422] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1181client # [ 0.589470] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker2 # [ 0.560577] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1183client # [ 0.589505] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1184worker2 # [ 0.560603] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1185lb # [ 0.581736] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1186client # [ 0.590295] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1187worker2 # [ 0.560626] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1188lb # [ 0.581981] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1189client # [ 0.590310] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1190lb # [ 0.582015] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1191worker2 # [ 0.560652] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1192worker2 # [ 0.560674] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1193client # [ 0.590316] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1194lb # [ 0.582540] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1195worker2 # [ 0.560698] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1196lb # [ 0.582741] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1197client # [ 0.590378] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1198worker2 # [ 0.560729] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1199lb # [ 0.582772] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200client # [ 0.590435] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1201worker2 # [ 0.560755] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1202lb # [ 0.583267] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1203client # [ 0.590490] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1204worker2 # [ 0.560781] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1205lb # [ 0.583572] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1206worker1 # [ 0.590127] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1207client # [ 0.590552] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1208lb # [ 0.583592] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1209worker2 # [ 0.560805] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1210worker1 # [ 0.590163] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1211worker2 # [ 0.560827] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1212lb # [ 0.583623] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1213client # [ 0.590607] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1214worker2 # [ 0.560851] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1215lb # [ 0.584120] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1216client # [ 0.590664] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1217worker1 # [ 0.590169] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1218worker2 # [ 0.560877] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1219worker1 # [ 0.590231] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1220lb # [ 0.584321] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1221client # [ 0.590721] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1222worker2 # [ 0.560902] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1223lb # [ 0.584339] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1224worker1 # [ 0.590287] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1225worker2 # [ 0.560925] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1226lb # [ 0.584372] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1227worker1 # [ 0.590344] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1228worker2 # [ 0.560949] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1229client # [ 0.590821] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1230client # [ 0.590880] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1231worker2 # [ 0.560974] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1232client # [ 0.590995] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1233worker2 # [ 0.561003] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1234client # [ 0.591129] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1235worker2 # [ 0.561013] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1236worker1 # [ 0.590395] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1237worker2 # [ 0.561018] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1238worker1 # [ 0.590446] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1239client # [ 0.591158] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1240worker1 # [ 0.590496] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1241worker1 # [ 0.590548] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1242worker2 # [ 0.561930] pci 0000:00:07.0: enabling device (0000 -> 0002)1243client # [ 0.591182] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1244client # [ 0.591207] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1245client # [ 0.591234] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1246worker1 # [ 0.590605] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1247worker1 # [ 0.590657] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1248client # [ 0.591259] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1249worker1 # [ 0.590771] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1250client # [ 0.591282] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1251worker1 # [ 0.590890] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1252client # [ 0.591316] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1253lb # [ 0.601388] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1254worker1 # [ 0.590918] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1255client # [ 0.591342] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1256worker1 # [ 0.590943] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1257lb # [ 0.601428] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1258client # [ 0.591367] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1259worker1 # [ 0.590967] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1260lb # [ 0.601435] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1261client # [ 0.591395] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1262worker1 # [ 0.590992] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1263lb # [ 0.601494] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1264client # [ 0.591425] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1265worker1 # [ 0.591016] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1266lb # [ 0.601545] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1267client # [ 0.591452] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1268worker1 # [ 0.591043] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1269client # [ 0.591475] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1270lb # [ 0.601599] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1271client # [ 0.591501] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1272lb # [ 0.601663] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1273client # [ 0.591526] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1274lb # [ 0.601715] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1275client # [ 0.591548] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1276worker1 # [ 0.591076] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1277lb # [ 0.601805] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1278worker1 # [ 0.591099] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1279client # [ 0.591573] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1280worker1 # [ 0.591125] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1281lb # [ 0.601859] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1282worker1 # [ 0.591151] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1283client # [ 0.591605] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1284worker1 # [ 0.591174] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1285client # [ 0.591618] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1286worker1 # [ 0.591222] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1287lb # [ 0.601939] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1288worker1 # [ 0.591243] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1289worker2 # [ 0.626224] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 62789 usecs1290client # [ 0.591623] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1291worker1 # [ 0.591265] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1292client # [ 0.592530] pci 0000:00:07.0: enabling device (0000 -> 0002)1293lb # [ 0.601989] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1294worker1 # [ 0.591294] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1295lb # [ 0.602174] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1296worker1 # [ 0.591315] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1297lb # [ 0.602229] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1298worker1 # [ 0.591339] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1299lb # [ 0.602253] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1300worker1 # [ 0.591368] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1301lb # [ 0.602316] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1302worker1 # [ 0.591378] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1303worker1 # [ 0.591383] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1304lb # [ 0.602341] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1305worker1 # [ 0.592258] pci 0000:00:07.0: enabling device (0000 -> 0002)1306lb # [ 0.602365] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1307lb # [ 0.602390] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1308lb # [ 0.602412] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1309lb # [ 0.602441] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1310lb # [ 0.602469] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1311lb # [ 0.602492] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1312lb # [ 0.602515] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1313lb # [ 0.602564] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1314lb # [ 0.602585] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1315lb # [ 0.602608] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1316lb # [ 0.602629] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1317lb # [ 0.602654] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1318lb # [ 0.602675] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1319lb # [ 0.602700] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1320lb # [ 0.602732] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1321lb # [ 0.602742] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1322lb # [ 0.602747] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1323lb # [ 0.603782] pci 0000:00:07.0: enabling device (0000 -> 0002)1324worker2 # [ 0.647993] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1325worker2 # [ 0.658424] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1326worker1 # [ 0.649625] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1327worker1 # [ 0.652273] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1328client # [ 0.654132] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1329client # [ 0.656731] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1330worker2 # [ 0.663970] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1331client # [ 0.668142] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1332worker2 # [ 0.670382] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333worker2 # [ 0.673702] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1334worker1 # [ 0.663291] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1335lb # [ 0.661952] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1336client # [ 0.674922] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1337worker2 # [ 0.684190] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1338lb # [ 0.672774] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1339lb # [ 0.675808] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1340client # [ 0.686085] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1341client # [ 0.688804] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1342worker1 # [ 0.674027] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1343worker1 # [ 0.677072] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1344worker1 # [ 0.679808] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1345worker2 # [ 0.695614] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1346lb # [ 0.686543] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1347worker1 # [ 0.690305] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1348client # [ 0.700241] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1349worker1 # [ 0.692815] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1350worker2 # [ 0.702299] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1351worker2 # [ 0.704903] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1352client # [ 0.703824] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1353worker1 # [ 0.703883] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1354worker2 # [ 0.715950] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1355lb # [ 0.697010] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1356lb # [ 0.701014] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1357lb # [ 0.703294] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1358client # [ 0.714061] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1359client # [ 0.716578] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1360worker2 # [ 0.723453] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1361worker1 # [ 0.714289] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1362lb # [ 0.713515] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1363lb # [ 0.716394] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1364client # [ 0.727844] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1365worker1 # [ 0.718657] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1366worker2 # [ 0.737176] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1367lb # [ 0.726807] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1368worker1 # [ 0.732629] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1369worker2 # [ 0.740178] msm_serial: driver initialized1370worker2 # [ 0.740350] SuperH (H)SCI(F) driver initialized1371worker2 # [ 0.740406] STM32 USART driver initialized1372client # [ 0.742918] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1373worker1 # [ 0.735734] msm_serial: driver initialized1374worker1 # [ 0.735893] SuperH (H)SCI(F) driver initialized1375worker1 # [ 0.735947] STM32 USART driver initialized1376client # [ 0.746389] msm_serial: driver initialized1377lb # [ 0.733499] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1378client # [ 0.746548] SuperH (H)SCI(F) driver initialized1379client # [ 0.746604] STM32 USART driver initialized1380lb # [ 0.747701] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1381lb # [ 0.750848] msm_serial: driver initialized1382lb # [ 0.750994] SuperH (H)SCI(F) driver initialized1383lb # [ 0.751053] STM32 USART driver initialized1384worker1 # [ 0.776664] loop: module loaded1385client # [ 0.784664] loop: module loaded1386worker1 # [ 0.776958] virtio_blk virtio2: 1/0/0 default/read/poll queues1387client # [ 0.784925] virtio_blk virtio2: 1/0/0 default/read/poll queues1388worker1 # [ 0.779455] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1389worker2 # [ 0.790486] loop: module loaded1390worker2 # [ 0.790746] virtio_blk virtio2: 1/0/0 default/read/poll queues1391worker2 # [ 0.791897] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1392client # [ 0.794188] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1393worker2 # [ 0.798900] megasas: 07.734.00.00-rc11394worker2 # [ 0.799770] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1395client # [ 0.798016] megasas: 07.734.00.00-rc11396client # [ 0.798854] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1397worker1 # [ 0.790415] megasas: 07.734.00.00-rc11398worker1 # [ 0.791527] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399lb # [ 0.795837] loop: module loaded1400lb # [ 0.796133] virtio_blk virtio2: 1/0/0 default/read/poll queues1401worker1 # [ 0.794141] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402worker1 # [ 0.794188] Intel/Sharp Extended Query Table at 0x00311403worker1 # [ 0.796396] Using buffer write method1404worker1 # [ 0.796523] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1405worker2 # [ 0.810067] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406client # [ 0.810654] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001407worker2 # [ 0.810188] Intel/Sharp Extended Query Table at 0x00311408client # [ 0.810710] Intel/Sharp Extended Query Table at 0x00311409worker2 # [ 0.813724] Using buffer write method1410client # [ 0.812949] Using buffer write method1411worker2 # [ 0.813853] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1412client # [ 0.813071] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1413lb # [ 0.805673] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1414worker2 # [ 0.815958] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415worker2 # [ 0.816000] Intel/Sharp Extended Query Table at 0x00311416worker1 # [ 0.810936] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417worker1 # [ 0.811048] Intel/Sharp Extended Query Table at 0x00311418worker1 # [ 0.813015] Using buffer write method1419worker1 # [ 0.813065] Concatenating MTD devices:1420worker1 # [ 0.813070] (0): "0.flash"1421worker1 # [ 0.813075] (1): "0.flash"1422worker1 # [ 0.813089] into device "0.flash"1423lb # [ 0.809610] megasas: 07.734.00.00-rc11424lb # [ 0.810456] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1425lb # [ 0.814613] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001426lb # [ 0.814662] Intel/Sharp Extended Query Table at 0x00311427worker2 # [ 0.834036] Using buffer write method1428worker2 # [ 0.834088] Concatenating MTD devices:1429worker2 # [ 0.834093] (0): "0.flash"1430worker2 # [ 0.834123] (1): "0.flash"1431worker2 # [ 0.834128] into device "0.flash"1432client # [ 0.831489] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001433client # [ 0.831561] Intel/Sharp Extended Query Table at 0x00311434client # [ 0.836389] Using buffer write method1435client # [ 0.836443] Concatenating MTD devices:1436client # [ 0.836448] (0): "0.flash"1437client # [ 0.836452] (1): "0.flash"1438client # [ 0.836468] into device "0.flash"1439lb # [ 0.832727] Using buffer write method1440lb # [ 0.832832] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1441lb # [ 0.835519] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001442lb # [ 0.835569] Intel/Sharp Extended Query Table at 0x00311443lb # [ 0.837982] Using buffer write method1444lb # [ 0.838021] Concatenating MTD devices:1445lb # [ 0.838026] (0): "0.flash"1446lb # [ 0.838031] (1): "0.flash"1447lb # [ 0.838047] into device "0.flash"1448client # [ 1.155680] Freeing initrd memory: 26972K1449worker2 # [ 1.159127] Freeing initrd memory: 26944K1450worker1 # [ 1.152023] Freeing initrd memory: 26976K1451worker2 # [ 1.165914] tun: Universal TUN/TAP device driver, 1.61452client # [ 1.162605] tun: Universal TUN/TAP device driver, 1.61453worker1 # [ 1.158919] tun: Universal TUN/TAP device driver, 1.61454client # [ 1.167268] thunder_xcv, ver 1.01455client # [ 1.167337] thunder_bgx, ver 1.01456client # [ 1.167359] nicpf, ver 1.01457client # [ 1.167968] e1000: Intel(R) PRO/1000 Network Driver1458client # [ 1.167977] e1000: Copyright (c) 1999-2006 Intel Corporation.1459worker2 # [ 1.170782] thunder_xcv, ver 1.01460client # [ 1.168001] e1000e: Intel(R) PRO/1000 Network Driver1461worker2 # [ 1.170836] thunder_bgx, ver 1.01462worker2 # [ 1.170859] nicpf, ver 1.01463worker1 # [ 1.163597] thunder_xcv, ver 1.01464client # [ 1.168012] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1465worker2 # [ 1.171470] e1000: Intel(R) PRO/1000 Network Driver1466lb # [ 1.160911] Freeing initrd memory: 26948K1467worker1 # [ 1.163659] thunder_bgx, ver 1.01468worker1 # [ 1.163683] nicpf, ver 1.01469client # [ 1.168042] igb: Intel(R) Gigabit Ethernet Network Driver1470worker2 # [ 1.171478] e1000: Copyright (c) 1999-2006 Intel Corporation.1471worker1 # [ 1.164278] e1000: Intel(R) PRO/1000 Network Driver1472client # [ 1.168047] igb: Copyright (c) 2007-2014 Intel Corporation.1473worker2 # [ 1.171504] e1000e: Intel(R) PRO/1000 Network Driver1474worker1 # [ 1.164287] e1000: Copyright (c) 1999-2006 Intel Corporation.1475client # [ 1.168074] igbvf: Intel(R) Gigabit Virtual Function Network Driver1476worker2 # [ 1.171514] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1477worker1 # [ 1.164313] e1000e: Intel(R) PRO/1000 Network Driver1478client # [ 1.168081] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1479worker2 # [ 1.171545] igb: Intel(R) Gigabit Ethernet Network Driver1480worker1 # [ 1.164324] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1481client # [ 1.168220] sky2: driver version 1.301482worker2 # [ 1.171551] igb: Copyright (c) 2007-2014 Intel Corporation.1483worker1 # [ 1.164357] igb: Intel(R) Gigabit Ethernet Network Driver1484worker2 # [ 1.171576] igbvf: Intel(R) Gigabit Virtual Function Network Driver1485worker1 # [ 1.164363] igb: Copyright (c) 2007-2014 Intel Corporation.1486lb # [ 1.167902] tun: Universal TUN/TAP device driver, 1.61487worker2 # [ 1.171583] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1488worker1 # [ 1.164414] igbvf: Intel(R) Gigabit Virtual Function Network Driver1489worker2 # [ 1.171723] sky2: driver version 1.301490client # [ 1.177825] usbcore: registered new interface driver usb-storage1491worker1 # [ 1.164420] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1492worker2 # [ 1.173398] usbcore: registered new interface driver usb-storage1493worker1 # [ 1.164565] sky2: driver version 1.301494client # [ 1.177899] usbcore: registered new interface driver usbserial_generic1495worker2 # [ 1.173548] usbcore: registered new interface driver usbserial_generic1496client # [ 1.177914] usbserial: USB Serial support registered for generic1497worker2 # [ 1.173563] usbserial: USB Serial support registered for generic1498client # [ 1.178521] hv_vmbus: registering driver hyperv_keyboard1499worker2 # [ 1.174938] ehci-pci 0000:00:07.0: EHCI Host Controller1500lb # [ 1.172623] thunder_xcv, ver 1.01501worker2 # [ 1.174973] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11502worker2 # [ 1.175221] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001503client # [ 1.183665] ehci-pci 0000:00:07.0: EHCI Host Controller1504worker1 # [ 1.174045] usbcore: registered new interface driver usb-storage1505client # [ 1.183711] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11506worker1 # [ 1.174221] usbcore: registered new interface driver usbserial_generic1507worker2 # [ 1.188056] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001508worker2 # [ 1.188451] hub 1-0:1.0: USB hub found1509worker2 # [ 1.188475] hub 1-0:1.0: 6 ports detected1510worker1 # [ 1.174238] usbserial: USB Serial support registered for generic1511worker1 # [ 1.174875] hv_vmbus: registering driver hyperv_keyboard1512worker1 # [ 1.176580] ehci-pci 0000:00:07.0: EHCI Host Controller1513lb # [ 1.173302] thunder_bgx, ver 1.01514worker2 # [ 1.191005] hv_vmbus: registering driver hyperv_keyboard1515worker1 # [ 1.176618] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11516lb # [ 1.173336] nicpf, ver 1.01517client # [ 1.183910] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001518client # [ 1.186950] rtc-pl031 9010000.pl031: registered as rtc01519worker1 # [ 1.176867] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001520lb # [ 1.173970] e1000: Intel(R) PRO/1000 Network Driver1521client # [ 1.186988] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T16:12:27 UTC (1790179947)1522lb # [ 1.173980] e1000: Copyright (c) 1999-2006 Intel Corporation.1523client # [ 1.187446] i2c_dev: i2c /dev entries driver1524worker2 # [ 1.192734] rtc-pl031 9010000.pl031: registered as rtc01525lb # [ 1.174005] e1000e: Intel(R) PRO/1000 Network Driver1526worker1 # [ 1.182686] rtc-pl031 9010000.pl031: registered as rtc01527lb # [ 1.174015] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1528worker2 # [ 1.192770] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T16:12:27 UTC (1790179947)1529lb # [ 1.174045] igb: Intel(R) Gigabit Ethernet Network Driver1530worker2 # [ 1.193104] i2c_dev: i2c /dev entries driver1531worker1 # [ 1.182724] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T16:12:27 UTC (1790179947)1532lb # [ 1.174051] igb: Copyright (c) 2007-2014 Intel Corporation.1533worker1 # [ 1.183221] i2c_dev: i2c /dev entries driver1534lb # [ 1.174075] igbvf: Intel(R) Gigabit Virtual Function Network Driver1535lb # [ 1.174081] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1536worker1 # [ 1.186418] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001537lb # [ 1.174221] sky2: driver version 1.301538worker1 # [ 1.186820] hub 1-0:1.0: USB hub found1539lb # [ 1.175861] usbcore: registered new interface driver usb-storage1540worker1 # [ 1.186843] hub 1-0:1.0: 6 ports detected1541lb # [ 1.176016] usbcore: registered new interface driver usbserial_generic1542client # [ 1.193337] sdhci: Secure Digital Host Controller Interface driver1543lb # [ 1.176032] usbserial: USB Serial support registered for generic1544client # [ 1.193349] sdhci: Copyright(c) Pierre Ossman1545lb # [ 1.177479] ehci-pci 0000:00:07.0: EHCI Host Controller1546client # [ 1.193614] Synopsys Designware Multimedia Card Interface Driver1547worker2 # [ 1.199101] sdhci: Secure Digital Host Controller Interface driver1548lb # [ 1.177512] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11549client # [ 1.193992] sdhci-pltfm: SDHCI platform and OF driver helper1550worker2 # [ 1.199118] sdhci: Copyright(c) Pierre Ossman1551lb # [ 1.178233] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001552worker2 # [ 1.199387] Synopsys Designware Multimedia Card Interface Driver1553client # [ 1.195612] hid: raw HID events driver (C) Jiri Kosina1554worker2 # [ 1.199756] sdhci-pltfm: SDHCI platform and OF driver helper1555worker1 # [ 1.191706] sdhci: Secure Digital Host Controller Interface driver1556lb # [ 1.190284] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001557client # [ 1.195875] usbcore: registered new interface driver usbhid1558client # [ 1.195883] usbhid: USB HID core driver1559worker2 # [ 1.201358] hid: raw HID events driver (C) Jiri Kosina1560lb # [ 1.190705] hub 1-0:1.0: USB hub found1561worker1 # [ 1.191723] sdhci: Copyright(c) Pierre Ossman1562client # [ 1.201746] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001563worker2 # [ 1.201608] usbcore: registered new interface driver usbhid1564lb # [ 1.190740] hub 1-0:1.0: 6 ports detected1565worker1 # [ 1.192001] Synopsys Designware Multimedia Card Interface Driver1566client # [ 1.202863] hub 1-0:1.0: USB hub found1567worker2 # [ 1.201616] usbhid: USB HID core driver1568worker1 # [ 1.192389] sdhci-pltfm: SDHCI platform and OF driver helper1569client # [ 1.203348] hub 1-0:1.0: 6 ports detected1570lb # [ 1.193235] hv_vmbus: registering driver hyperv_keyboard1571worker1 # [ 1.196825] hid: raw HID events driver (C) Jiri Kosina1572client # [ 1.204336] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1573worker1 # [ 1.197109] usbcore: registered new interface driver usbhid1574worker1 # [ 1.197116] usbhid: USB HID core driver1575lb # [ 1.194942] rtc-pl031 9010000.pl031: registered as rtc01576worker2 # [ 1.207513] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1577lb # [ 1.194982] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T16:12:27 UTC (1790179947)1578worker2 # [ 1.209126] drop_monitor: Initializing network drop monitor service1579client # [ 1.207207] drop_monitor: Initializing network drop monitor service1580lb # [ 1.195331] i2c_dev: i2c /dev entries driver1581worker2 # [ 1.209327] NET: Registered PF_INET6 protocol family1582client # [ 1.207386] NET: Registered PF_INET6 protocol family1583client # [ 1.209519] Segment Routing with IPv61584worker2 # [ 1.212968] Segment Routing with IPv61585worker1 # [ 1.200183] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1586client # [ 1.209546] In-situ OAM (IOAM) with IPv61587worker2 # [ 1.213006] In-situ OAM (IOAM) with IPv61588client # [ 1.209582] NET: Registered PF_PACKET protocol family1589worker2 # [ 1.213038] NET: Registered PF_PACKET protocol family1590worker1 # [ 1.203155] drop_monitor: Initializing network drop monitor service1591worker1 # [ 1.203384] NET: Registered PF_INET6 protocol family1592worker1 # [ 1.205579] Segment Routing with IPv61593client # [ 1.211366] 9pnet: Installing 9P2000 support1594worker1 # [ 1.205615] In-situ OAM (IOAM) with IPv61595client # [ 1.211426] Key type dns_resolver registered1596worker1 # [ 1.205678] NET: Registered PF_PACKET protocol family1597worker2 # [ 1.214883] 9pnet: Installing 9P2000 support1598lb # [ 1.201147] sdhci: Secure Digital Host Controller Interface driver1599worker2 # [ 1.214940] Key type dns_resolver registered1600lb # [ 1.201163] sdhci: Copyright(c) Pierre Ossman1601lb # [ 1.201438] Synopsys Designware Multimedia Card Interface Driver1602lb # [ 1.201827] sdhci-pltfm: SDHCI platform and OF driver helper1603worker1 # [ 1.207562] 9pnet: Installing 9P2000 support1604worker1 # [ 1.207624] Key type dns_resolver registered1605lb # [ 1.203435] hid: raw HID events driver (C) Jiri Kosina1606lb # [ 1.203687] usbcore: registered new interface driver usbhid1607lb # [ 1.203696] usbhid: USB HID core driver1608client # [ 1.218538] registered taskstats version 11609client # [ 1.218722] Loading compiled-in X.509 certificates1610worker2 # [ 1.222048] registered taskstats version 11611lb # [ 1.209942] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1612worker2 # [ 1.222235] Loading compiled-in X.509 certificates1613lb # [ 1.211653] drop_monitor: Initializing network drop monitor service1614lb # [ 1.211833] NET: Registered PF_INET6 protocol family1615worker1 # [ 1.214957] registered taskstats version 11616lb # [ 1.215646] Segment Routing with IPv61617worker1 # [ 1.215115] Loading compiled-in X.509 certificates1618lb # [ 1.215684] In-situ OAM (IOAM) with IPv61619lb # [ 1.215714] NET: Registered PF_PACKET protocol family1620worker2 # [ 1.231599] Demotion targets for Node 0: null1621worker2 # [ 1.231744] Key type .fscrypt registered1622worker2 # [ 1.231751] Key type fscrypt-provisioning registered1623worker2 # [ 1.231868] ima: No TPM chip found, activating TPM-bypass!1624lb # [ 1.217598] 9pnet: Installing 9P2000 support1625lb # [ 1.217654] Key type dns_resolver registered1626worker2 # [ 1.231892] ima: Allocated hash algorithm: sha11627worker2 # [ 1.231917] ima: No architecture policies found1628client # [ 1.228261] Demotion targets for Node 0: null1629client # [ 1.228421] Key type .fscrypt registered1630client # [ 1.228429] Key type fscrypt-provisioning registered1631client # [ 1.228544] ima: No TPM chip found, activating TPM-bypass!1632client # [ 1.228566] ima: Allocated hash algorithm: sha11633client # [ 1.228592] ima: No architecture policies found1634worker1 # [ 1.224532] Demotion targets for Node 0: null1635worker1 # [ 1.224693] Key type .fscrypt registered1636client # [ 1.233196] input: gpio-keys as /devices/platform/gpio-keys/input/input01637worker1 # [ 1.224700] Key type fscrypt-provisioning registered1638worker2 # [ 1.236871] input: gpio-keys as /devices/platform/gpio-keys/input/input01639worker1 # [ 1.224823] ima: No TPM chip found, activating TPM-bypass!1640lb # [ 1.224497] registered taskstats version 11641worker1 # [ 1.224846] ima: Allocated hash algorithm: sha11642worker1 # [ 1.224871] ima: No architecture policies found1643worker1 # [ 1.229633] input: gpio-keys as /devices/platform/gpio-keys/input/input01644lb # [ 1.225281] Loading compiled-in X.509 certificates1645lb # [ 1.233984] Demotion targets for Node 0: null1646lb # [ 1.234134] Key type .fscrypt registered1647lb # [ 1.234142] Key type fscrypt-provisioning registered1648lb # [ 1.234265] ima: No TPM chip found, activating TPM-bypass!1649lb # [ 1.234287] ima: Allocated hash algorithm: sha11650lb # [ 1.234313] ima: No architecture policies found1651lb # [ 1.239151] input: gpio-keys as /devices/platform/gpio-keys/input/input01652client # [ 1.256728] clk: Disabling unused clocks1653client # [ 1.256778] PM: genpd: Disabling unused power domains1654worker2 # [ 1.260330] clk: Disabling unused clocks1655worker2 # [ 1.260378] PM: genpd: Disabling unused power domains1656worker1 # [ 1.253355] clk: Disabling unused clocks1657worker1 # [ 1.253407] PM: genpd: Disabling unused power domains1658client # [ 1.261330] Freeing unused kernel memory: 4736K1659worker2 # [ 1.265036] Freeing unused kernel memory: 4736K1660client # [ 1.261623] Run /init as init process1661worker2 # [ 1.265250] Run /init as init process1662worker1 # [ 1.258029] Freeing unused kernel memory: 4736K1663worker1 # [ 1.258274] Run /init as init process1664lb # [ 1.261971] clk: Disabling unused clocks1665lb # [ 1.262018] PM: genpd: Disabling unused power domains1666lb # [ 1.266584] Freeing unused kernel memory: 4736K1667lb # [ 1.266877] Run /init as init process1668client # [ 1.280048] systemd[1]: Successfully made /usr/ read-only.1669worker2 # [ 1.284670] systemd[1]: Successfully made /usr/ read-only.1670worker1 # [ 1.276313] systemd[1]: Successfully made /usr/ read-only.1671lb # [ 1.283488] systemd[1]: Successfully made /usr/ read-only.1672worker2 # [ 1.434130] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker1 # [ 1.433569] usb 1-1: new high-speed USB device number 2 using ehci-pci1674lb # [ 1.436902] usb 1-1: new high-speed USB device number 2 using ehci-pci1675client # [ 1.449350] usb 1-1: new high-speed USB device number 2 using ehci-pci1676worker2 # [ 1.590976] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677lb # [ 1.589318] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11678worker1 # [ 1.588584] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679client # [ 1.608783] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11680worker2 # [ 1.619831] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1681client # [ 1.616442] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1682worker2 # [ 1.633722] systemd[1]: Detected virtualization qemu.1683worker1 # [ 1.611536] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1684worker2 # [ 1.636352] systemd[1]: Detected architecture arm64.1685worker1 # [ 1.625661] systemd[1]: Detected virtualization qemu.1686client # [ 1.631020] systemd[1]: Detected virtualization qemu.1687lb # [ 1.618928] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1688worker2 # [ 1.638595] systemd[1]: Running in initrd.1689lb # [ 1.626062] systemd[1]: Detected virtualization qemu.1690worker1 # [ 1.628057] systemd[1]: Detected architecture arm64.1691client # [ 1.631620] systemd[1]: Detected architecture arm64.1692lb # [ 1.627201] systemd[1]: Detected architecture arm64.1693lb # [ 1.628319] systemd[1]: Running in initrd.1694worker1 # [ 1.630243] systemd[1]: Running in initrd.1695client # [ 1.631677] systemd[1]: Running in initrd.1696worker2 # [ 1.641538] systemd[1]: Initializing machine ID from random generator.1697lb # [ 1.629853] systemd[1]: Initializing machine ID from random generator.1698lb # [ 1.631418] systemd[1]: Hostname set to <lb>.1699client # [ 1.632833] systemd[1]: Initializing machine ID from random generator.1700worker1 # [ 1.633161] systemd[1]: Initializing machine ID from random generator.1701worker2 # [ 1.645076] systemd[1]: Hostname set to <worker2>.1702client # [ 1.633171] systemd[1]: Hostname set to <client>.1703worker1 # [ 1.636517] systemd[1]: Hostname set to <worker1>.1704worker2 # [ 1.686412] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705worker1 # [ 1.681932] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706lb # [ 1.684981] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707client # [ 1.701654] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708worker2 # [ 1.810043] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker1 # [ 1.805561] usb 1-2: new high-speed USB device number 3 using ehci-pci1710lb # [ 1.804771] usb 1-2: new high-speed USB device number 3 using ehci-pci1711client # [ 1.825289] usb 1-2: new high-speed USB device number 3 using ehci-pci1712client # [ 1.947446] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713lb # [ 1.939055] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714worker2 # [ 1.961092] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker2 # [ 1.970500] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21716worker2 # [ 1.974720] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01717worker1 # [ 1.965409] systemd[1]: bpf-restrict-fs: LSM BPF program attached1718worker1 # [ 1.971490] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719client # [ 1.983118] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21720lb # [ 1.971591] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721client # [ 1.986336] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722worker1 # [ 1.977414] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723lb # [ 1.976978] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01724client # [ 2.023983] systemd[1]: Queued start job for default target Initrd Default Target.1725client # [ 2.036460] systemd[1]: Created slice Slice /system/modprobe.1726client # [ 2.037735] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1727client # [ 2.039032] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728client # [ 2.039993] systemd[1]: Reached target Path Units.1729client # [ 2.040861] systemd[1]: Reached target Slice Units.1730client # [ 2.041625] systemd[1]: Reached target Swaps.1731client # [ 2.042263] systemd[1]: Reached target Timer Units.1732client # [ 2.043220] systemd[1]: Listening on D-Bus System Message Bus Socket.1733client # [ 2.044357] systemd[1]: Listening on Journal Socket (/dev/log).1734client # [ 2.045448] systemd[1]: Listening on Journal Sockets.1735client # [ 2.046386] systemd[1]: Listening on udev Control Socket.1736client # [ 2.047376] systemd[1]: Listening on udev Kernel Socket.1737client # [ 2.048181] systemd[1]: Reached target Socket Units.1738lb # [ 2.040439] systemd[1]: Queued start job for default target Initrd Default Target.1739client # [ 2.051244] systemd[1]: Starting Create List of Static Device Nodes...1740worker2 # [ 2.055221] systemd[1]: Queued start job for default target Initrd Default Target.1741client # [ 2.052397] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1742lb # [ 2.047071] systemd[1]: Created slice Slice /system/modprobe.1743lb # [ 2.048303] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1744worker2 # [ 2.064201] systemd[1]: Created slice Slice /system/modprobe.1745client # [ 2.061576] systemd[1]: Mounting Kernel Configuration File System...1746lb # [ 2.049686] systemd[1]: Expecting device /dev/disk/by-label/nixos...1747worker2 # [ 2.065307] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1748lb # [ 2.049727] systemd[1]: Reached target Path Units.1749lb # [ 2.049748] systemd[1]: Reached target Slice Units.1750lb # [ 2.049768] systemd[1]: Reached target Swaps.1751lb # [ 2.049787] systemd[1]: Reached target Timer Units.1752worker2 # [ 2.066610] systemd[1]: Expecting device /dev/disk/by-label/nixos...1753worker2 # [ 2.066647] systemd[1]: Reached target Path Units.1754lb # [ 2.049995] systemd[1]: Listening on D-Bus System Message Bus Socket.1755worker2 # [ 2.066664] systemd[1]: Reached target Slice Units.1756lb # [ 2.050143] systemd[1]: Listening on Journal Socket (/dev/log).1757worker2 # [ 2.066684] systemd[1]: Reached target Swaps.1758lb # [ 2.050277] systemd[1]: Listening on Journal Sockets.1759worker2 # [ 2.066702] systemd[1]: Reached target Timer Units.1760lb # [ 2.050400] systemd[1]: Listening on udev Control Socket.1761worker2 # [ 2.066911] systemd[1]: Listening on D-Bus System Message Bus Socket.1762lb # [ 2.050512] systemd[1]: Listening on udev Kernel Socket.1763client # [ 2.069609] systemd[1]: Starting Journal Service...1764worker2 # [ 2.067046] systemd[1]: Listening on Journal Socket (/dev/log).1765lb # [ 2.050535] systemd[1]: Reached target Socket Units.1766worker2 # [ 2.067165] systemd[1]: Listening on Journal Sockets.1767worker2 # [ 2.067273] systemd[1]: Listening on udev Control Socket.1768worker2 # [ 2.067373] systemd[1]: Listening on udev Kernel Socket.1769worker2 # [ 2.067390] systemd[1]: Reached target Socket Units.1770lb # [ 2.061246] systemd[1]: Starting Create List of Static Device Nodes...1771worker2 # [ 2.077538] systemd[1]: Starting Create List of Static Device Nodes...1772lb # [ 2.061344] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1773worker2 # [ 2.078682] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1774worker1 # [ 2.074013] systemd[1]: Queued start job for default target Initrd Default Target.1775lb # [ 2.072943] systemd[1]: Mounting Kernel Configuration File System...1776worker2 # [ 2.089843] systemd[1]: Mounting Kernel Configuration File System...1777worker1 # [ 2.084025] systemd[1]: Created slice Slice /system/modprobe.1778lb # [ 2.081830] systemd[1]: Starting Journal Service...1779worker1 # [ 2.085297] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1780client # [ 2.095264] systemd[1]: Starting Load Kernel Modules...1781client # [ 2.096300] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1782worker1 # [ 2.086791] systemd[1]: Expecting device /dev/disk/by-label/nixos...1783worker1 # [ 2.086836] systemd[1]: Reached target Path Units.1784worker1 # [ 2.086857] systemd[1]: Reached target Slice Units.1785worker1 # [ 2.086879] systemd[1]: Reached target Swaps.1786worker1 # [ 2.086898] systemd[1]: Reached target Timer Units.1787worker1 # [ 2.087126] systemd[1]: Listening on D-Bus System Message Bus Socket.1788worker1 # [ 2.087312] systemd[1]: Listening on Journal Socket (/dev/log).1789worker1 # [ 2.087451] systemd[1]: Listening on Journal Sockets.1790worker1 # [ 2.087584] systemd[1]: Listening on udev Control Socket.1791worker1 # [ 2.087708] systemd[1]: Listening on udev Kernel Socket.1792worker1 # [ 2.087731] systemd[1]: Reached target Socket Units.1793worker1 # [ 2.099691] systemd[1]: Starting Create List of Static Device Nodes...1794worker1 # [ 2.100902] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1795worker2 # [ 2.114370] systemd[1]: Starting Journal Service...1796client # [ 2.117577] systemd[1]: Starting Coldplug All udev Devices...1797worker1 # [ 2.113100] systemd[1]: Mounting Kernel Configuration File System...1798worker2 # [ 2.125848] systemd[1]: Starting Load Kernel Modules...1799lb # [ 2.112994] systemd[1]: Starting Load Kernel Modules...1800client # [ 2.123562] systemd-journald[72]: Collecting audit messages is disabled.1801worker2 # [ 2.127054] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1802lb # [ 2.114017] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1803client # [ 2.126194] systemd[1]: Finished Create List of Static Device Nodes.1804lb # [ 2.121239] systemd[1]: Starting Coldplug All udev Devices...1805worker2 # [ 2.139676] systemd[1]: Starting Coldplug All udev Devices...1806lb # [ 2.129153] systemd[1]: Finished Create List of Static Device Nodes.1807client # [ 2.147258] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1808worker1 # [ 2.142153] systemd[1]: Starting Journal Service...1809worker2 # [ 2.151806] systemd[1]: Finished Create List of Static Device Nodes.1810worker2 # [ 2.152772] systemd[1]: Mounted Kernel Configuration File System.1811worker1 # [ 2.145304] systemd[1]: Starting Load Kernel Modules...1812worker1 # [ 2.145443] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1813lb # [ 2.141300] systemd-journald[72]: Collecting audit messages is disabled.1814worker2 # [ 2.158649] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1815worker2 # [ 2.161708] systemd-journald[72]: Collecting audit messages is disabled.1816client # [ 2.157879] systemd[1]: Mounted Kernel Configuration File System.1817lb # [ 2.158304] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1818worker1 # [ 2.161788] systemd[1]: Starting Coldplug All udev Devices...1819lb # [ 2.160132] systemd[1]: Mounted Kernel Configuration File System.1820worker1 # [ 2.181704] systemd[1]: Finished Create List of Static Device Nodes.1821worker1 # [ 2.182837] systemd[1]: Mounted Kernel Configuration File System.1822worker1 # [ 2.194061] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1823lb # [ 2.190802] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1824client # [ 2.210108] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1825client # [ 2.213442] systemd[1]: Starting Create Static Device Nodes in /dev...1826lb # [ 2.213050] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1827lb # [ 2.217636] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1828client # [ 2.228611] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1829worker2 # [ 2.231173] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1830worker1 # [ 2.228407] systemd-journald[72]: Collecting audit messages is disabled.1831worker2 # [ 2.246749] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1832client # [ 2.245342] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1833lb # [ 2.234668] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01834worker2 # [ 2.249448] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1835lb # [ 2.234970] [drm] features: -virgl +edid -resource_blob -host_visible1836lb # [ 2.234984] [drm] features: -context_init1837lb # [ 2.235830] [drm] number of scanouts: 11838lb # [ 2.235853] [drm] number of cap sets: 01839lb # [ 2.240138] systemd[1]: Starting Create Static Device Nodes in /dev...1840worker2 # [ 2.254561] systemd[1]: Starting Create Static Device Nodes in /dev...1841worker1 # [ 2.248465] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1842worker2 # [ 2.265372] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01843worker2 # [ 2.265669] [drm] features: -virgl +edid -resource_blob -host_visible1844worker2 # [ 2.265680] [drm] features: -context_init1845lb # [ 2.254321] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1846lb # [ 2.254353] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01847client # [ 2.267422] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01848client # [ 2.267735] [drm] features: -virgl +edid -resource_blob -host_visible1849client # [ 2.267749] [drm] features: -context_init1850client # [ 2.268614] [drm] number of scanouts: 11851client # [ 2.268638] [drm] number of cap sets: 01852worker1 # [ 2.265603] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1853worker1 # [ 2.270322] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1854worker1 # [ 2.273382] systemd[1]: Starting Create Static Device Nodes in /dev...1855worker2 # [ 2.286852] [drm] number of scanouts: 11856worker2 # [ 2.286892] [drm] number of cap sets: 01857worker2 # [ 2.288853] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1858worker2 # [ 2.288870] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01859client # [ 2.289692] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1860client # [ 2.289721] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01861worker1 # [ 2.287517] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01862worker1 # [ 2.287795] [drm] features: -virgl +edid -resource_blob -host_visible1863client # [ 2.302088] systemd[1]: Finished Create Static Device Nodes in /dev.1864worker1 # [ 2.287809] [drm] features: -context_init1865worker1 # [ 2.288633] [drm] number of scanouts: 11866worker1 # [ 2.288652] [drm] number of cap sets: 01867lb # [ 2.285245] Console: switching to colour frame buffer device 160x501868client # [ 2.305583] systemd[1]: Reached target Preparation for Local File Systems.1869client # [ 2.305628] systemd[1]: Reached target Local File Systems.1870client # [ 2.313006] systemd[1]: Starting Rule-based Manager for Device Events and Files...1871client # [ 2.315809] systemd[1]: Started Journal Service.1872worker1 # [ 2.306049] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1873worker1 # [ 2.306078] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01874client # [ 2.304694] systemd-modules-load[74]: Inserted module 'dm_mod'1875client # [ 2.306239] systemd-modules-load[74]: Module 'virtio_balloon' is built in1876client # [ 2.307552] systemd-modules-load[74]: Module 'virtio_console' is built in1877lb # [ 2.313465] systemd[1]: Finished Create Static Device Nodes in /dev.1878lb # [ 2.313717] systemd[1]: Reached target Preparation for Local File Systems.1879lb # [ 2.313744] systemd[1]: Reached target Local File Systems.1880lb # [ 2.299118] systemd-modules-load[73]: Inserted module 'dm_mod'1881lb # [ 2.321400] systemd[1]: Starting Rule-based Manager for Device Events and Files...1882worker2 # [ 2.326449] Console: switching to colour frame buffer device 160x501883lb # [ 2.322673] systemd[1]: Started Journal Service.1884lb # [ 2.323965] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1885client # [ 2.329735] Console: switching to colour frame buffer device 160x501886client # [ 2.335400] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1887lb # [ 2.307548] systemd-modules-load[73]: Module 'virtio_balloon' is built in1888worker2 # [ 2.346302] systemd[1]: Started Journal Service.1889lb # [ 2.316587] systemd-modules-load[73]: Module 'virtio_console' is built in1890lb # [ 2.318202] systemd-modules-load[73]: Inserted module 'virtio_gpu'1891lb # [ 2.319319] systemd-modules-load[73]: Module 'virtio_rng' is built in1892client # [ 2.334525] systemd[1]: Starting Create System Files and Directories...1893worker2 # [ 2.354210] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1894worker2 # [ 2.331798] systemd-modules-load[73]: Inserted module 'dm_mod'1895worker2 # [ 2.339850] systemd-modules-load[73]: Module 'virtio_balloon' is built in1896lb # [ 2.325403] systemd[1]: Finished Load Kernel Modules.1897client # [ 2.339537] systemd-modules-load[74]: Inserted module 'virtio_gpu'1898worker1 # [ 2.341868] Console: switching to colour frame buffer device 160x501899worker2 # [ 2.343947] systemd-modules-load[73]: Module 'virtio_console' is built in1900client # [ 2.344363] systemd-modules-load[74]: Module 'virtio_rng' is built in1901worker1 # [ 2.352028] systemd[1]: Finished Create Static Device Nodes in /dev.1902worker1 # [ 2.352271] systemd[1]: Reached target Preparation for Local File Systems.1903worker1 # [ 2.352298] systemd[1]: Reached target Local File Systems.1904worker2 # [ 2.352677] systemd[1]: Finished Create Static Device Nodes in /dev.1905worker2 # [ 2.353904] systemd[1]: Reached target Preparation for Local File Systems.1906worker1 # [ 2.360780] systemd[1]: Starting Rule-based Manager for Device Events and Files...1907worker2 # [ 2.355087] systemd[1]: Reached target Local File Systems.1908client # [ 2.358193] systemd[1]: Finished Load Kernel Modules.1909worker2 # [ 2.364321] systemd[1]: Starting Create System Files and Directories...1910lb # [ 2.349795] systemd[1]: Starting Apply Kernel Variables...1911worker2 # [ 2.366735] systemd-modules-load[73]: Inserted module 'virtio_gpu'1912lb # [ 2.352660] systemd[1]: Starting Create System Files and Directories...1913worker2 # [ 2.370683] systemd-modules-load[73]: Module 'virtio_rng' is built in1914worker1 # [ 2.378256] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1915worker2 # [ 2.376322] systemd[1]: Starting Rule-based Manager for Device Events and Files...1916worker2 # [ 2.378387] systemd[1]: Finished Load Kernel Modules.1917client # [ 2.385798] systemd[1]: Starting Apply Kernel Variables...1918worker1 # [ 2.393906] systemd[1]: Finished Load Kernel Modules.1919worker1 # [ 2.397993] systemd[1]: Starting Apply Kernel Variables...1920worker2 # [ 2.400274] systemd[1]: Starting Apply Kernel Variables...1921client # [ 2.402042] systemd[1]: Finished Create System Files and Directories.1922client # [ 2.426500] systemd[1]: Finished Apply Kernel Variables.1923lb # [ 2.419847] systemd[1]: Finished Create System Files and Directories.1924worker1 # [ 2.424434] systemd-modules-load[73]: Inserted module 'dm_mod'1925lb # [ 2.424564] systemd-udevd[79]: Using default interface naming scheme 'v261'.1926worker2 # [ 2.441392] systemd[1]: Finished Create System Files and Directories.1927worker1 # [ 2.447386] systemd[1]: Started Journal Service.1928worker1 # [ 2.429673] systemd-modules-load[73]: Module 'virtio_balloon' is built in1929lb # [ 2.429034] systemd[1]: Finished Apply Kernel Variables.1930worker1 # [ 2.430913] systemd-modules-load[73]: Module 'virtio_console' is built in1931client # [ 2.449364] systemd-udevd[79]: Using default interface naming scheme 'v261'.1932worker1 # [ 2.444304] systemd-modules-load[73]: Inserted module 'virtio_gpu'1933worker1 # [ 2.448479] systemd-modules-load[73]: Module 'virtio_rng' is built in1934worker2 # [ 2.466465] systemd[1]: Finished Apply Kernel Variables.1935worker1 # [ 2.452381] systemd[1]: Starting Create System Files and Directories...1936worker2 # [ 2.469628] systemd-udevd[79]: Using default interface naming scheme 'v261'.1937worker1 # [ 2.458305] systemd[1]: Finished Apply Kernel Variables.1938lb # [ 2.457924] systemd[1]: Started Rule-based Manager for Device Events and Files.1939worker1 # [ 2.468647] systemd-udevd[78]: Using default interface naming scheme 'v261'.1940client # [ 2.484221] systemd[1]: Started Rule-based Manager for Device Events and Files.1941worker2 # [ 2.501300] systemd[1]: Started Rule-based Manager for Device Events and Files.1942worker1 # [ 2.512969] systemd[1]: Finished Create System Files and Directories.1943worker1 # [ 2.517108] systemd[1]: Started Rule-based Manager for Device Events and Files.1944lb # [ 2.520952] systemd[1]: Starting Virtual Console Setup...1945client # [ 2.560172] systemd[1]: Starting Virtual Console Setup...1946worker2 # [ 2.576211] systemd[1]: Starting Virtual Console Setup...1947lb # [ 2.580582] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1948worker1 # [ 2.584776] systemd[1]: Starting Virtual Console Setup...1949lb # [ 2.584759] systemd[1]: Finished Virtual Console Setup.1950client # [ 2.616588] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1951client # [ 2.621087] systemd[1]: Finished Virtual Console Setup.1952worker2 # [ 2.624551] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1953worker2 # [ 2.628732] systemd[1]: Finished Virtual Console Setup.1954worker1 # [ 2.652612] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1955worker1 # [ 2.656728] systemd[1]: Finished Virtual Console Setup.1956lb # [ 3.204498] systemd[1]: Finished Coldplug All udev Devices.1957worker2 # [ 3.221142] systemd[1]: Finished Coldplug All udev Devices.1958lb # [ 3.205911] systemd[1]: Reached target System Initialization.1959lb # [ 3.208452] systemd[1]: Reached target Basic System.1960worker2 # [ 3.225498] systemd[1]: Reached target System Initialization.1961worker2 # [ 3.226862] systemd[1]: Reached target Basic System.1962client # [ 3.251062] systemd[1]: Finished Coldplug All udev Devices.1963client # [ 3.252113] systemd[1]: Reached target System Initialization.1964client # [ 3.252964] systemd[1]: Reached target Basic System.1965worker1 # [ 3.250557] systemd[1]: Finished Coldplug All udev Devices.1966worker1 # [ 3.251567] systemd[1]: Reached target System Initialization.1967worker1 # [ 3.252701] systemd[1]: Reached target Basic System.1968worker2 # [ 3.386271] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1969lb # [ 3.400766] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1970worker1 # [ 3.404432] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971worker2 # [ 3.420395] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972lb # [ 3.404347] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973worker2 # [ 3.423802] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1974lb # [ 3.408923] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1975worker1 # [ 3.439597] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1976worker1 # [ 3.444387] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1977client # [ 3.457996] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1978client # [ 3.469546] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1979client # [ 3.472588] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1980lb # [ 3.480160] systemd[1]: Found device /dev/disk/by-label/nixos.1981worker2 # [ 3.500530] systemd[1]: Found device /dev/disk/by-label/nixos.1982worker2 # [ 3.507566] systemd[1]: Reached target Initrd Root Device.1983lb # [ 3.491795] systemd[1]: Reached target Initrd Root Device.1984lb # [ 3.498036] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985worker2 # [ 3.515158] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986client # [ 3.544371] systemd[1]: Found device /dev/disk/by-label/nixos.1987worker1 # [ 3.533176] systemd[1]: Found device /dev/disk/by-label/nixos.1988worker1 # [ 3.540112] systemd[1]: Reached target Initrd Root Device.1989client # [ 3.552625] systemd[1]: Reached target Initrd Root Device.1990worker1 # [ 3.545931] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1991client # [ 3.558283] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992worker2 # [ 3.567770] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1993lb # [ 3.554872] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1994worker2 # [ 3.578797] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995lb # [ 3.563114] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1996worker2 # [ 3.584262] systemd[1]: Mounting /sysroot...1997lb # [ 3.568220] systemd[1]: Mounting /sysroot...1998worker1 # [ 3.596200] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1999worker1 # [ 3.603576] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000client # [ 3.620874] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 1,952 hidden lines…
3953client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds3954client # farm-top> LOG-farm-top-t13955worker1 # [ 60.416713] nix-daemon[856]: reaped child process 954, status = succeeded3956worker1 # [ 60.501302] nix-daemon[856]: accepted connection from pid 970, user nix-grpc-daemon (trusted)3957worker1 # [ 60.522016] nix-daemon[856]: reaped child process 973, status = succeeded3958lb # [ 60.529436] niks3-server[985]: 2026/09/23 16:13:26 INFO Received push request method=POST path=/api/pushes3959worker1 # [ 60.550505] nix-grpc-daemon[870]: time=2026-09-23T16:13:26.887Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"3960worker1 # [ 60.552070] nix-grpc-daemon[870]: time=2026-09-23T16:13:26.887Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"3961worker1 # [ 60.591129] nix-grpc-daemon[870]: time=2026-09-23T16:13:26.929Z level=INFO msg="Uploading 1 narinfos"3962lb # [ 60.582044] niks3-server[985]: 2026/09/23 16:13:26 INFO Received sign narinfos request method=POST path=/api/pushes/6/sign3963lb # [ 60.595901] niks3-server[985]: 2026/09/23 16:13:26 INFO Signed narinfos id=6 count=13964lb # [ 60.604366] niks3-server[985]: 2026/09/23 16:13:26 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls3965lb # [ 60.618072] niks3-server[985]: 2026/09/23 16:13:26 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst3966lb # [ 60.627683] niks3-server[985]: 2026/09/23 16:13:26 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3967lb # [ 60.632239] niks3-server[985]: 2026/09/23 16:13:26 INFO Received complete push request method=POST path=/api/pushes/6/complete3968worker1 # [ 60.638415] nix-grpc-daemon[870]: time=2026-09-23T16:13:26.977Z level=INFO msg="Upload complete. (255ms)"3969lb # [ 60.639606] niks3-server[985]: 2026/09/23 16:13:26 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo3970worker1 # [ 60.653224] nix-grpc-daemon[487]: ts=2026-09-23T16:13:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=13971worker1 # [ 60.673029] nix-daemon[856]: reaped child process 952, status = succeeded3972client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 5.41 seconds)3973client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath3974client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds)3975client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top3976client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)3977client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top3978client # copying 3 paths...3979client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...3980client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...3981client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...3982client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)3983(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.99 seconds)3984subtest: repeat is answered Cached by the scheduler without building3985worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top3986worker1 # finding garbage collector roots...3987worker1 # deleting specified paths...3988worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'3989worker1 # deleting unused links...3990worker1 # note: hard linking is currently saving 0.0 KiB3991worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)3992worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top3993worker2 # finding garbage collector roots...3994worker2 # deleting specified paths...3995worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'3996worker2 # deleting unused links...3997worker2 # note: hard linking is currently saving 0.0 KiB3998worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)3999client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24000client # don't know how to build these paths:4001client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4002worker2 # [ 61.995986] nix-grpc-daemon[786]: ts=2026-09-23T16:13:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375944003worker2 # [ 62.004105] nix-grpc-daemon[786]: ts=2026-09-23T16:13:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944004worker1 # [ 62.041426] nix-grpc-daemon[487]: ts=2026-09-23T16:13:28Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14005client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.34 seconds)4006client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4007client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds)4008lb: must succeed: curl -sf localhost:9901/clusters4009lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4010lb: must succeed: curl -sf localhost:9901/clusters4011lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4012worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4013worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4014(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.24 seconds)4015subtest: inputs already in the cache are not sent to the scheduler4016lb: must succeed: curl -sf localhost:9901/clusters4017lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4018lb: must succeed: curl -sf localhost:9901/clusters4019lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4020worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4021worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4022client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24023client # don't know how to build these paths:4024client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4025worker1 # [ 62.915734] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395544026worker1 # [ 62.924656] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544027worker1 # [ 62.933850] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.269Z level=INFO msg="All 2 paths already cached"4028worker1 # [ 62.946480] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04029worker1 # [ 62.953819] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14030worker1 # [ 62.963083] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544031worker1 # [ 62.975092] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544032worker1 # [ 62.990209] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.328Z level=INFO msg="All 2 paths already cached"4033client # copying 1 paths to 'lb:50051'...4034client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4035worker1 # [ 63.002304] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395544036worker1 # [ 63.017659] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4037worker1 # [ 63.167444] nix-daemon[856]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4038worker1 # [ 63.188963] nix-daemon[856]: reaped child process 1005, status = succeeded4039lb # [ 63.191172] niks3-server[985]: 2026/09/23 16:13:29 INFO Received push request method=POST path=/api/pushes4040worker1 # [ 63.203520] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.541Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4041worker1 # [ 63.205274] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.541Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4042lb # [ 63.221419] niks3-server[985]: 2026/09/23 16:13:29 INFO Received sign narinfos request method=POST path=/api/pushes/7/sign4043worker1 # [ 63.231743] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.570Z level=INFO msg="Uploading 1 narinfos"4044lb # [ 63.232069] niks3-server[985]: 2026/09/23 16:13:29 INFO Signed narinfos id=7 count=14045lb # [ 63.244566] niks3-server[985]: 2026/09/23 16:13:29 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4046lb # [ 63.253757] niks3-server[985]: 2026/09/23 16:13:29 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4047lb # [ 63.264612] niks3-server[985]: 2026/09/23 16:13:29 INFO Received complete push request method=POST path=/api/pushes/7/complete4048lb # [ 63.271904] niks3-server[985]: 2026/09/23 16:13:29 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4049worker1 # [ 63.274068] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.612Z level=INFO msg="Upload complete. (240ms)"4050worker1 # [ 63.285451] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=7524051worker1 # [ 63.300705] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544052worker1 # [ 63.309223] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4053worker1 # [ 63.320337] nix-daemon[856]: reaped child process 1000, status = succeeded4054client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4055client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4056worker1 # [ 63.331937] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4057client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4058client # farm-top> LOG-farm-top-t1b4059worker1 # [ 64.641484] nix-daemon[856]: reaped child process 1009, status = succeeded4060worker1 # [ 64.712742] nix-daemon[856]: accepted connection from pid 1025, user nix-grpc-daemon (trusted)4061worker1 # [ 64.734640] nix-daemon[856]: reaped child process 1028, status = succeeded4062lb # [ 64.739631] niks3-server[985]: 2026/09/23 16:13:31 INFO Received push request method=POST path=/api/pushes4063worker1 # [ 64.754794] nix-grpc-daemon[870]: time=2026-09-23T16:13:31.090Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4064worker1 # [ 64.757889] nix-grpc-daemon[870]: time=2026-09-23T16:13:31.090Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4065lb # [ 64.785776] niks3-server[985]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4066worker1 # [ 64.800111] nix-grpc-daemon[870]: time=2026-09-23T16:13:31.138Z level=INFO msg="Uploading 1 narinfos"4067lb # [ 64.801053] niks3-server[985]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4068lb # [ 64.809962] niks3-server[985]: 2026/09/23 16:13:31 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign4069lb # [ 64.817429] niks3-server[985]: 2026/09/23 16:13:31 INFO Signed narinfos id=8 count=14070lb # [ 64.827631] niks3-server[985]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4071lb # [ 64.835761] niks3-server[985]: 2026/09/23 16:13:31 INFO Received complete push request method=POST path=/api/pushes/8/complete4072worker1 # [ 64.843629] nix-grpc-daemon[870]: time=2026-09-23T16:13:31.182Z level=INFO msg="Upload complete. (300ms)"4073lb # [ 64.844128] niks3-server[985]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4074worker1 # [ 64.857178] nix-grpc-daemon[487]: ts=2026-09-23T16:13:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14075worker1 # [ 64.878532] nix-daemon[856]: reaped child process 1007, status = succeeded4076client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.21 seconds)4077client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4078client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds)4079lb: must succeed: curl -sf localhost:9901/clusters4080lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4081lb: must succeed: curl -sf localhost:9901/clusters4082lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4083worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4084worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4085(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.74 seconds)4086subtest: outputs read back through any worker carry the cache signature4087client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24088client # don't know how to build these paths:4089client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4090worker2 # [ 65.487008] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375944091worker2 # [ 65.495187] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944092worker1 # [ 65.499094] nix-grpc-daemon[487]: ts=2026-09-23T16:13:31Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04093worker1 # [ 65.503979] nix-grpc-daemon[487]: ts=2026-09-23T16:13:31Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14094worker2 # [ 65.521340] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944095worker2 # [ 65.537166] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944096client # copying 1 paths to 'lb:50051'...4097client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4098worker2 # [ 65.551407] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375944099worker2 # [ 65.566915] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4100worker2 # [ 65.600154] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=info event=niks3_push_started pid=8694101worker2 # [ 65.767617] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.092Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4102worker2 # [ 65.881104] nix-daemon[844]: accepted connection from pid 882, user nix-grpc-daemon (trusted)4103worker2 # [ 65.901615] nix-daemon[844]: reaped child process 885, status = succeeded4104lb # [ 65.891034] niks3-server[985]: 2026/09/23 16:13:32 INFO Received push request method=POST path=/api/pushes4105worker2 # [ 65.919001] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.242Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4106worker2 # [ 65.923078] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.242Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4107lb # [ 65.933270] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4108worker2 # [ 65.965857] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.289Z level=INFO msg="Uploading 1 narinfos"4109lb # [ 65.942484] niks3-server[985]: 2026/09/23 16:13:32 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4110lb # [ 65.955746] niks3-server[985]: 2026/09/23 16:13:32 INFO Signed narinfos id=9 count=14111lb # [ 65.963093] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4112lb # [ 65.977780] niks3-server[985]: 2026/09/23 16:13:32 INFO Received complete push request method=POST path=/api/pushes/9/complete4113worker2 # [ 66.002583] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.327Z level=INFO msg="Upload complete. (230ms)"4114lb # [ 65.988103] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4115worker2 # [ 66.013431] nix-grpc-daemon[786]: ts=2026-09-23T16:13:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=5124116worker2 # [ 66.026731] nix-grpc-daemon[786]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944117worker2 # [ 66.033132] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4118worker2 # [ 66.038051] nix-daemon[844]: reaped child process 867, status = succeeded4119client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4120worker2 # [ 66.050612] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4121client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4122client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4123client # farm-a> LOG-farm-a-sig4124worker2 # [ 66.478388] nix-daemon[844]: reaped child process 889, status = succeeded4125worker2 # [ 66.576683] nix-daemon[844]: accepted connection from pid 905, user nix-grpc-daemon (trusted)4126worker2 # [ 66.598658] nix-daemon[844]: reaped child process 908, status = succeeded4127lb # [ 66.580225] niks3-server[985]: 2026/09/23 16:13:32 INFO Received push request method=POST path=/api/pushes4128worker2 # [ 66.607633] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.930Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4129worker2 # [ 66.610257] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.930Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4130lb # [ 66.621934] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4131lb # [ 66.637073] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4132lb # [ 66.650646] niks3-server[985]: 2026/09/23 16:13:32 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4133worker2 # [ 66.675601] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.000Z level=INFO msg="Uploading 1 narinfos"4134lb # [ 66.662741] niks3-server[985]: 2026/09/23 16:13:32 INFO Signed narinfos id=10 count=14135lb # [ 66.669722] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4136lb # [ 66.681142] niks3-server[985]: 2026/09/23 16:13:33 INFO Received complete push request method=POST path=/api/pushes/10/complete4137worker2 # [ 66.705580] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.030Z level=INFO msg="Upload complete. (246ms)"4138lb # [ 66.691755] niks3-server[985]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4139worker1 # [ 66.710853] nix-grpc-daemon[487]: ts=2026-09-23T16:13:33Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04140worker1 # [ 66.717293] nix-grpc-daemon[487]: ts=2026-09-23T16:13:33Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14141worker2 # [ 66.721245] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14142worker2 # [ 66.740656] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944143worker2 # [ 66.747752] nix-daemon[844]: reaped child process 887, status = succeeded4144worker2 # [ 66.756769] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944145worker2 # [ 66.771368] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.095Z level=INFO msg="All 1 paths already cached"4146client # copying 1 paths to 'lb:50051'...4147client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4148worker2 # [ 66.785617] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375944149worker2 # [ 66.799661] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4150worker2 # [ 66.952347] nix-daemon[844]: accepted connection from pid 912, user nix-grpc-daemon (trusted)4151worker2 # [ 66.975623] nix-daemon[844]: reaped child process 915, status = succeeded4152lb # [ 66.966098] niks3-server[985]: 2026/09/23 16:13:33 INFO Received push request method=POST path=/api/pushes4153worker2 # [ 67.085472] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.409Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4154worker2 # [ 67.090387] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.409Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4155lb # [ 67.098420] niks3-server[985]: 2026/09/23 16:13:33 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4156worker2 # [ 67.123057] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.447Z level=INFO msg="Uploading 1 narinfos"4157lb # [ 67.116578] niks3-server[985]: 2026/09/23 16:13:33 INFO Signed narinfos id=11 count=14158lb # [ 67.119950] niks3-server[985]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4159lb # [ 67.142164] niks3-server[985]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4160worker2 # [ 67.166659] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.491Z level=INFO msg="Upload complete. (352ms)"4161lb # [ 67.155030] niks3-server[985]: 2026/09/23 16:13:33 INFO Received complete push request method=POST path=/api/pushes/11/complete4162lb # [ 67.160097] niks3-server[985]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4163worker2 # [ 67.180771] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=6244164worker2 # [ 67.201094] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944165worker2 # [ 67.211937] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4166worker2 # [ 67.219983] nix-daemon[844]: reaped child process 910, status = succeeded4167client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4168client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4169worker2 # [ 67.239900] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4170client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4171client # farm-b> LOG-farm-b-sig4172worker2 # [ 68.924402] nix-daemon[844]: reaped child process 919, status = succeeded4173worker2 # [ 69.043504] nix-daemon[844]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4174worker2 # [ 69.065976] nix-daemon[844]: reaped child process 938, status = succeeded4175lb # [ 69.060151] niks3-server[985]: 2026/09/23 16:13:35 INFO Received push request method=POST path=/api/pushes4176worker2 # [ 69.086317] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.409Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4177worker2 # [ 69.088259] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.409Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4178lb # [ 69.105768] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4179worker2 # [ 69.134490] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.459Z level=INFO msg="Uploading 1 narinfos"4180lb # [ 69.118907] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4181lb # [ 69.127973] niks3-server[985]: 2026/09/23 16:13:35 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4182lb # [ 69.130770] niks3-server[985]: 2026/09/23 16:13:35 INFO Signed narinfos id=12 count=14183lb # [ 69.139914] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4184lb # [ 69.149108] niks3-server[985]: 2026/09/23 16:13:35 INFO Received complete push request method=POST path=/api/pushes/12/complete4185worker2 # [ 69.165771] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.490Z level=INFO msg="Upload complete. (309ms)"4186lb # [ 69.153466] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4187worker2 # [ 69.181486] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14188worker1 # [ 69.173643] nix-grpc-daemon[487]: ts=2026-09-23T16:13:35Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04189worker1 # [ 69.174008] nix-grpc-daemon[487]: ts=2026-09-23T16:13:35Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14190worker2 # [ 69.204529] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944191worker2 # [ 69.213555] nix-daemon[844]: reaped child process 917, status = succeeded4192worker2 # [ 69.222853] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944193worker2 # [ 69.237262] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.562Z level=INFO msg="All 2 paths already cached"4194client # copying 1 paths to 'lb:50051'...4195worker2 # [ 69.248504] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375944196client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4197worker2 # [ 69.271797] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4198lb # [ 69.257226] envoy[895]: conn peer=[2001:db8:1::1]:48994 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9502 tx=4373 ms=52324199lb # [ 69.260895] envoy[895]: conn peer=[2001:db8:1::1]:38990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=844200lb # [ 69.263820] envoy[895]: conn peer=[2001:db8:1::1]:38992 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1895 ms=19624201worker2 # [ 69.444138] nix-daemon[844]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4202worker2 # [ 69.466582] nix-daemon[844]: reaped child process 945, status = succeeded4203lb # [ 69.457181] niks3-server[985]: 2026/09/23 16:13:35 INFO Received push request method=POST path=/api/pushes4204worker2 # [ 69.483658] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.808Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4205worker2 # [ 69.485336] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.808Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4206lb # [ 69.489323] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4207worker2 # [ 69.515443] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.840Z level=INFO msg="Uploading 1 narinfos"4208lb # [ 69.502760] niks3-server[985]: 2026/09/23 16:13:35 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4209lb # [ 69.515584] niks3-server[985]: 2026/09/23 16:13:35 INFO Signed narinfos id=13 count=14210lb # [ 69.521138] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4211lb # [ 69.532504] niks3-server[985]: 2026/09/23 16:13:35 INFO Received complete push request method=POST path=/api/pushes/13/complete4212lb # [ 69.540279] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4213worker2 # [ 69.556658] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.881Z level=INFO msg="Upload complete. (265ms)"4214worker2 # [ 69.567454] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=7524215worker2 # [ 69.580734] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944216worker2 # [ 69.591707] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4217worker2 # [ 69.597778] nix-daemon[844]: reaped child process 940, status = succeeded4218worker2 # [ 69.608609] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4219client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4220client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4221client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4222client # farm-top> LOG-farm-top-sig4223worker2 # [ 70.845004] nix-daemon[844]: reaped child process 949, status = succeeded4224worker2 # [ 70.970678] nix-daemon[844]: accepted connection from pid 965, user nix-grpc-daemon (trusted)4225worker2 # [ 70.991636] nix-daemon[844]: reaped child process 968, status = succeeded4226lb # [ 70.975535] niks3-server[985]: 2026/09/23 16:13:37 INFO Received push request method=POST path=/api/pushes4227worker2 # [ 71.004429] nix-grpc-daemon[869]: time=2026-09-23T16:13:37.327Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4228worker2 # [ 71.005949] nix-grpc-daemon[869]: time=2026-09-23T16:13:37.327Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4229lb # [ 71.023817] niks3-server[985]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4230worker2 # [ 71.054812] nix-grpc-daemon[869]: time=2026-09-23T16:13:37.379Z level=INFO msg="Uploading 1 narinfos"4231lb # [ 71.039792] niks3-server[985]: 2026/09/23 16:13:37 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4232lb # [ 71.048799] niks3-server[985]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4233lb # [ 71.059091] niks3-server[985]: 2026/09/23 16:13:37 INFO Signed narinfos id=14 count=14234lb # [ 71.070825] niks3-server[985]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4235lb # [ 71.078921] niks3-server[985]: 2026/09/23 16:13:37 INFO Received complete push request method=POST path=/api/pushes/14/complete4236worker2 # [ 71.100501] nix-grpc-daemon[869]: time=2026-09-23T16:13:37.425Z level=INFO msg="Upload complete. (278ms)"4237lb # [ 71.083001] niks3-server[985]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4238worker2 # [ 71.113361] nix-grpc-daemon[786]: ts=2026-09-23T16:13:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14239worker2 # [ 71.132884] nix-daemon[844]: reaped child process 947, status = succeeded4240client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.89 seconds)4241client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4242client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4243client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4244worker1 # [ 71.466617] nix-grpc-daemon[487]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:582044245client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4246client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4247client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4248client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4249worker1 # [ 71.501071] nix-grpc-daemon[487]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:582044250worker1 # [ 71.510277] nix-grpc-daemon[487]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4251worker1 # [ 71.533693] nix-grpc-daemon[487]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4252worker1 # [ 71.548453] nix-grpc-daemon[487]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4253client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.26 seconds)4254client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4255worker2 # [ 71.785457] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:473624256worker2 # [ 71.793854] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:473624257client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.21 seconds)4258client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4259client # finding garbage collector roots...4260client # removing stale temporary roots file "/nix/var/nix/temproots/897"4261client # removing stale temporary roots file "/nix/var/nix/temproots/917"4262client # removing stale temporary roots file "/nix/var/nix/temproots/849"4263client # removing stale temporary roots file "/nix/var/nix/temproots/905"4264client # removing stale temporary roots file "/nix/var/nix/temproots/940"4265client # removing stale temporary roots file "/nix/var/nix/temproots/885"4266client # removing stale temporary roots file "/nix/var/nix/temproots/925"4267client # removing stale temporary roots file "/nix/var/nix/temproots/864"4268client # deleting specified paths...4269client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4270client # deleting unused links...4271client # note: hard linking is currently saving 0.0 KiB4272worker2 # [ 72.126057] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375944273worker1 # [ 72.122104] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544274worker1 # [ 72.137872] nix-grpc-daemon[870]: time=2026-09-23T16:13:38.474Z level=INFO msg="All 1 paths already cached"4275worker2 # [ 72.155839] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375944276worker2 # [ 72.166318] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375944277worker2 # [ 72.176198] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375944278client # copying 3 paths...4279worker2 # [ 72.191429] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375944280client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4281worker2 # [ 72.218371] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375944282worker2 # [ 72.225648] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_out=2324283worker2 # [ 72.242224] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375944284worker2 # [ 72.249252] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_out=1764285worker2 # [ 72.258355] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375944286client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4287worker2 # [ 72.263468] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_out=1284288client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4289client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.49 seconds)4290(finished: subtest: outputs read back through any worker carry the cache signature, in 7.04 seconds)4291subtest: two clients wanting the same drv build it once4292client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4293client # [ 72.346712] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4294client # Running as unit: dedup1.service; invocation ID: c09a6918455c425fb56f9eee49bf339d4295client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.06 seconds)4296client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4297worker2 # [ 72.533417] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375944298client # [ 72.535909] nix[993]: don't know how to build these paths:4299client # [ 72.537301] nix[993]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4300worker1 # [ 72.536831] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04301worker1 # [ 72.544941] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14302worker1 # [ 72.552346] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544303worker1 # [ 72.563550] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544304client # [ 72.590417] nix[993]: copying 1 paths to 'lb:50051'...4305client # [ 72.592142] nix[993]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4306worker1 # [ 72.579655] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395544307worker1 # [ 72.595628] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4308worker1 # [ 72.739122] nix-daemon[856]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)4309worker1 # [ 72.759903] nix-daemon[856]: reaped child process 1051, status = succeeded4310lb # [ 72.763183] niks3-server[985]: 2026/09/23 16:13:39 INFO Received push request method=POST path=/api/pushes4311worker1 # [ 72.776084] nix-grpc-daemon[870]: time=2026-09-23T16:13:39.114Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4312worker1 # [ 72.777571] nix-grpc-daemon[870]: time=2026-09-23T16:13:39.114Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4313lb # [ 72.796761] niks3-server[985]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4314worker1 # [ 72.812155] nix-grpc-daemon[870]: time=2026-09-23T16:13:39.150Z level=INFO msg="Uploading 1 narinfos"4315lb # [ 72.811623] niks3-server[985]: 2026/09/23 16:13:39 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4316lb # [ 72.822799] niks3-server[985]: 2026/09/23 16:13:39 INFO Signed narinfos id=15 count=14317lb # [ 72.829030] niks3-server[985]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4318worker1 # [ 72.847119] nix-grpc-daemon[870]: time=2026-09-23T16:13:39.185Z level=INFO msg="Upload complete. (232ms)"4319lb # [ 72.844986] niks3-server[985]: 2026/09/23 16:13:39 INFO Received complete push request method=POST path=/api/pushes/15/complete4320lb # [ 72.853827] niks3-server[985]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4321worker1 # [ 72.854702] nix-grpc-daemon[487]: ts=2026-09-23T16:13:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=4324322worker1 # [ 72.871345] nix-grpc-daemon[487]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544323worker1 # [ 72.878991] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4324worker1 # [ 72.885059] nix-daemon[856]: reaped child process 1046, status = succeeded4325client # [ 72.907944] nix[993]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4326worker1 # [ 72.900196] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4327client # [ 73.435773] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4328client # Running as unit: dedup2.service; invocation ID: bb6de19e44dd4f1f8001a38a6f9d122a4329client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4330worker2 # [ 73.673181] nix-grpc-daemon[786]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375944331client # [ 73.678449] nix[1009]: don't know how to build these paths:4332client # [ 73.679538] nix[1009]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4333worker1 # [ 73.676604] nix-grpc-daemon[487]: ts=2026-09-23T16:13:40Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04334client # [ 73.697372] nix[1009]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4335worker1 # [ 73.684848] nix-grpc-daemon[487]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544336client: must succeed: sleep 24337client: (finished: must succeed: sleep 2, in 2.05 seconds)4338worker1 # [ 76.927232] nix-daemon[856]: reaped child process 1055, status = succeeded4339client: waiting for success: ! systemctl is-active dedup1 dedup24340worker1 # [ 77.028099] nix-daemon[856]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)4341worker1 # [ 77.045308] nix-daemon[856]: reaped child process 1090, status = succeeded4342lb # [ 77.049167] niks3-server[985]: 2026/09/23 16:13:43 INFO Received push request method=POST path=/api/pushes4343worker1 # [ 77.059187] nix-grpc-daemon[870]: time=2026-09-23T16:13:43.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4344worker1 # [ 77.061657] nix-grpc-daemon[870]: time=2026-09-23T16:13:43.395Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4345lb # [ 77.088107] niks3-server[985]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4346worker1 # [ 77.104610] nix-grpc-daemon[870]: time=2026-09-23T16:13:43.443Z level=INFO msg="Uploading 1 narinfos"4347lb # [ 77.103071] niks3-server[985]: 2026/09/23 16:13:43 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4348lb # [ 77.110593] niks3-server[985]: 2026/09/23 16:13:43 INFO Signed narinfos id=16 count=14349lb # [ 77.115361] niks3-server[985]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4350lb # [ 77.127969] niks3-server[985]: 2026/09/23 16:13:43 INFO Received complete push request method=POST path=/api/pushes/16/complete4351lb # [ 77.131349] niks3-server[985]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4352worker1 # [ 77.141942] nix-grpc-daemon[870]: time=2026-09-23T16:13:43.480Z level=INFO msg="Upload complete. (225ms)"4353lb # [ 77.142575] niks3-server[985]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4354worker1 # [ 77.157100] nix-grpc-daemon[487]: ts=2026-09-23T16:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14355worker1 # [ 77.175114] nix-grpc-daemon[487]: ts=2026-09-23T16:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14356worker1 # [ 77.181343] nix-daemon[856]: reaped child process 1053, status = succeeded4357client # [ 77.224144] systemd[1]: dedup2.service: Deactivated successfully.4358client # [ 77.228185] systemd[1]: dedup2.service: Consumed 208ms CPU time over 3.792s wall clock time, 35.2M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic.4359client # [ 77.235985] systemd[1]: dedup1.service: Deactivated successfully.4360client # [ 77.240759] systemd[1]: dedup1.service: Consumed 169ms CPU time over 4.892s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.4361client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4362client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4363client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4364worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4365worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4366worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4367worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4368worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4369worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4370worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4371worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4372(finished: subtest: two clients wanting the same drv build it once, in 6.07 seconds)4373subtest: --timeout and --max-silent-time reach the builder4374worker2 # [ 78.602461] nix-grpc-daemon[786]: ts=2026-09-23T16:13:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375944375worker1 # [ 78.606309] nix-grpc-daemon[487]: ts=2026-09-23T16:13:44Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04376worker1 # [ 78.612706] nix-grpc-daemon[487]: ts=2026-09-23T16:13:44Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14377worker2 # [ 78.630605] nix-grpc-daemon[786]: ts=2026-09-23T16:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944378worker2 # [ 78.645066] nix-grpc-daemon[786]: ts=2026-09-23T16:13:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944379worker2 # [ 78.659088] nix-grpc-daemon[786]: ts=2026-09-23T16:13:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375944380worker2 # [ 78.667211] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4381worker2 # [ 78.795358] nix-daemon[844]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4382worker2 # [ 78.817212] nix-daemon[844]: reaped child process 996, status = succeeded4383lb # [ 78.805392] niks3-server[985]: 2026/09/23 16:13:45 INFO Received push request method=POST path=/api/pushes4384worker2 # [ 78.832427] nix-grpc-daemon[869]: time=2026-09-23T16:13:45.156Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4385worker2 # [ 78.834629] nix-grpc-daemon[869]: time=2026-09-23T16:13:45.157Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4386lb # [ 78.841762] niks3-server[985]: 2026/09/23 16:13:45 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4387worker2 # [ 78.866790] nix-grpc-daemon[869]: time=2026-09-23T16:13:45.191Z level=INFO msg="Uploading 1 narinfos"4388lb # [ 78.856006] niks3-server[985]: 2026/09/23 16:13:45 INFO Signed narinfos id=17 count=14389lb # [ 78.863490] niks3-server[985]: 2026/09/23 16:13:45 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4390lb # [ 78.879123] niks3-server[985]: 2026/09/23 16:13:45 INFO Received complete push request method=POST path=/api/pushes/17/complete4391lb # [ 78.887745] niks3-server[985]: 2026/09/23 16:13:45 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4392worker2 # [ 78.910748] nix-grpc-daemon[869]: time=2026-09-23T16:13:45.235Z level=INFO msg="Upload complete. (233ms)"4393lb # [ 78.894824] niks3-server[985]: 2026/09/23 16:13:45 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4394worker2 # [ 78.921338] nix-grpc-daemon[786]: ts=2026-09-23T16:13:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=4244395worker2 # [ 78.936490] nix-grpc-daemon[786]: ts=2026-09-23T16:13:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944396worker2 # [ 78.944835] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4397worker2 # [ 78.955210] nix-daemon[844]: reaped child process 991, status = succeeded4398worker2 # [ 78.962933] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4399lb # [ 79.258489] envoy[895]: conn peer=[2001:db8:1::1]:39004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9541 tx=4382 ms=56414400lb # [ 79.264445] envoy[895]: conn peer=[2001:db8:1::1]:49408 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=724401lb # [ 79.269590] envoy[895]: conn peer=[2001:db8:1::1]:49422 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=564402lb # [ 79.275242] envoy[895]: conn peer=[2001:db8:1::1]:49394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=1674403lb # [ 79.281297] envoy[895]: conn peer=[2001:db8:1::1]:49432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=374404lb # [ 79.286368] envoy[895]: conn peer=[2001:db8:1::1]:49452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=35154405lb # [ 79.292194] envoy[895]: conn peer=[2001:db8:1::1]:49444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3044 tx=1563 ms=46514406worker2 # [ 84.777264] nix-grpc-daemon[786]: ts=2026-09-23T16:13:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04407worker2 # [ 84.802006] nix-daemon[844]: reaped child process 998, status = succeeded4408worker2 # [ 84.803823] nix-daemon[844]: reaped child process 1000, status = succeeded4409worker2 # [ 85.023126] nix-grpc-daemon[786]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375944410worker1 # [ 85.028425] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04411worker1 # [ 85.028750] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14412worker1 # [ 85.041885] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544413worker1 # [ 85.058370] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544414worker1 # [ 85.070246] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395544415worker1 # [ 85.085388] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4416worker1 # [ 85.240597] nix-daemon[856]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4417worker1 # [ 85.257466] nix-daemon[856]: reaped child process 1110, status = succeeded4418lb # [ 85.266481] niks3-server[985]: 2026/09/23 16:13:51 INFO Received push request method=POST path=/api/pushes4419worker1 # [ 85.276667] nix-grpc-daemon[870]: time=2026-09-23T16:13:51.614Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4420worker1 # [ 85.278160] nix-grpc-daemon[870]: time=2026-09-23T16:13:51.614Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4421lb # [ 85.296818] niks3-server[985]: 2026/09/23 16:13:51 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4422worker1 # [ 85.305305] nix-grpc-daemon[870]: time=2026-09-23T16:13:51.643Z level=INFO msg="Uploading 1 narinfos"4423lb # [ 85.307873] niks3-server[985]: 2026/09/23 16:13:51 INFO Signed narinfos id=18 count=14424lb # [ 85.325723] niks3-server[985]: 2026/09/23 16:13:51 INFO Received complete push request method=POST path=/api/pushes/18/complete4425lb # [ 85.335375] niks3-server[985]: 2026/09/23 16:13:51 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4426worker1 # [ 85.342105] nix-grpc-daemon[870]: time=2026-09-23T16:13:51.680Z level=INFO msg="Upload complete. (241ms)"4427lb # [ 85.342142] niks3-server[985]: 2026/09/23 16:13:51 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4428lb # [ 85.348560] niks3-server[985]: 2026/09/23 16:13:51 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4429worker1 # [ 85.352300] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=4244430worker1 # [ 85.368737] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544431worker1 # [ 85.376358] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4432worker1 # [ 85.384397] nix-daemon[856]: reaped child process 1105, status = succeeded4433worker1 # [ 85.394685] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4434lb # [ 89.260828] envoy[895]: conn peer=[2001:db8:1::1]:49456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3014 tx=978 ms=62034435worker1 # [ 91.409310] nix-grpc-daemon[487]: ts=2026-09-23T16:13:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04436worker1 # [ 91.431555] nix-daemon[856]: reaped child process 1114, status = succeeded4437worker1 # [ 91.433206] nix-daemon[856]: reaped child process 1112, status = succeeded4438(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.19 seconds)4439subtest: the first client leaving does not fail the second4440client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4441client # [ 91.627381] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4442client # Running as unit: share1.service; invocation ID: 971a6aba2c7e4663976ca2f65ce3b22f4443client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.08 seconds)4444client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4445worker1 # [ 91.822942] nix-grpc-daemon[487]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395544446client # [ 91.842292] nix[1081]: don't know how to build these paths:4447client # [ 91.843376] nix[1081]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4448worker1 # [ 91.843224] nix-grpc-daemon[487]: ts=2026-09-23T16:13:58Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04449worker1 # [ 91.850461] nix-grpc-daemon[487]: ts=2026-09-23T16:13:58Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14450worker2 # [ 91.868774] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944451worker2 # [ 91.881308] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944452client # [ 91.891901] nix[1081]: copying 1 paths to 'lb:50051'...4453client # [ 91.894043] nix[1081]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4454worker2 # [ 91.894478] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375944455worker2 # [ 91.909821] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4456worker2 # [ 92.067685] nix-daemon[844]: accepted connection from pid 1023, user nix-grpc-daemon (trusted)4457worker2 # [ 92.087307] nix-daemon[844]: reaped child process 1026, status = succeeded4458lb # [ 92.075987] niks3-server[985]: 2026/09/23 16:13:58 INFO Received push request method=POST path=/api/pushes4459worker2 # [ 92.103411] nix-grpc-daemon[869]: time=2026-09-23T16:13:58.427Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4460worker2 # [ 92.105224] nix-grpc-daemon[869]: time=2026-09-23T16:13:58.427Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4461lb # [ 92.120797] niks3-server[985]: 2026/09/23 16:13:58 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4462worker2 # [ 92.146529] nix-grpc-daemon[869]: time=2026-09-23T16:13:58.471Z level=INFO msg="Uploading 1 narinfos"4463lb # [ 92.134479] niks3-server[985]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4464lb # [ 92.143006] niks3-server[985]: 2026/09/23 16:13:58 INFO Signed narinfos id=19 count=14465lb # [ 92.151845] niks3-server[985]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4466lb # [ 92.156861] niks3-server[985]: 2026/09/23 16:13:58 INFO Received complete push request method=POST path=/api/pushes/19/complete4467lb # [ 92.161501] niks3-server[985]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4468worker2 # [ 92.177226] nix-grpc-daemon[869]: time=2026-09-23T16:13:58.502Z level=INFO msg="Upload complete. (249ms)"4469worker2 # [ 92.188510] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=4324470worker2 # [ 92.204785] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944471worker2 # [ 92.213510] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4472worker2 # [ 92.219594] nix-daemon[844]: reaped child process 1021, status = succeeded4473worker2 # [ 92.230148] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4474client # [ 92.227070] nix[1081]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4475client # [ 92.730436] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4476client # Running as unit: share2.service; invocation ID: 20c68947242647e480b0157788aee4b74477client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4478client # [ 92.946470] nix[1097]: don't know how to build these paths:4479client # [ 92.948526] nix[1097]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4480worker1 # [ 92.933635] nix-grpc-daemon[487]: ts=2026-09-23T16:13:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395544481worker1 # [ 92.947405] nix-grpc-daemon[487]: ts=2026-09-23T16:13:59Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04482worker2 # [ 92.965703] nix-grpc-daemon[786]: ts=2026-09-23T16:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944483client # [ 92.966619] nix[1097]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4484client: must succeed: sleep 2; systemctl kill -s INT share14485client # [ 96.136824] systemd[1]: share1.service: Sent signal SIGINT to main process 1081 (nix) on client request.4486client # [ 96.147291] nix[1081]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4487client # [ 96.149525] nix[1081]: error: interrupted by the user4488client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4489client: waiting for success: ! systemctl is-active share14490client # [ 96.217890] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4491client # [ 96.220085] systemd[1]: share1.service: Failed with result 'exit-code'.4492client # [ 96.223231] systemd[1]: share1.service: Consumed 177ms CPU time over 4.595s wall clock time, 39.2M memory peak, 6.2K incoming IP traffic, 8K outgoing IP traffic.4493client: (finished: waiting for success: ! systemctl is-active share1, in 1.11 seconds)4494client: waiting for success: ! systemctl is-active share24495worker2 # [ 97.737660] nix-daemon[844]: reaped child process 1030, status = succeeded4496worker2 # [ 97.806615] nix-daemon[844]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4497worker2 # [ 97.826554] nix-daemon[844]: reaped child process 1067, status = succeeded4498lb # [ 97.815984] niks3-server[985]: 2026/09/23 16:14:04 INFO Received push request method=POST path=/api/pushes4499worker2 # [ 97.847871] nix-grpc-daemon[869]: time=2026-09-23T16:14:04.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4500worker2 # [ 97.849729] nix-grpc-daemon[869]: time=2026-09-23T16:14:04.169Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4501lb # [ 97.869031] niks3-server[985]: 2026/09/23 16:14:04 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4502lb # [ 97.882633] niks3-server[985]: 2026/09/23 16:14:04 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign4503worker2 # [ 97.903853] nix-grpc-daemon[869]: time=2026-09-23T16:14:04.228Z level=INFO msg="Uploading 1 narinfos"4504lb # [ 97.887204] niks3-server[985]: 2026/09/23 16:14:04 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4505lb # [ 97.894545] niks3-server[985]: 2026/09/23 16:14:04 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4506lb # [ 97.902309] niks3-server[985]: 2026/09/23 16:14:04 INFO Signed narinfos id=20 count=14507lb # [ 97.913149] niks3-server[985]: 2026/09/23 16:14:04 INFO Received complete push request method=POST path=/api/pushes/20/complete4508worker2 # [ 97.934847] nix-grpc-daemon[869]: time=2026-09-23T16:14:04.259Z level=INFO msg="Upload complete. (285ms)"4509lb # [ 97.921145] niks3-server[985]: 2026/09/23 16:14:04 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4510worker2 # [ 97.948494] nix-grpc-daemon[786]: ts=2026-09-23T16:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14511worker2 # [ 97.966001] nix-grpc-daemon[786]: ts=2026-09-23T16:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14512worker2 # [ 97.972527] nix-daemon[844]: reaped child process 1028, status = succeeded4513client # [ 97.978718] systemd[1]: share2.service: Deactivated successfully.4514client # [ 97.984244] systemd[1]: share2.service: Consumed 188ms CPU time over 5.249s wall clock time, 33.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4515client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4516client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4517client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4518(finished: subtest: the first client leaving does not fail the second, in 7.14 seconds)4519subtest: build hook: nix-daemon with builders = grpc://lb4520client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4521client # [ 98.837917] systemd[1]: Started Nix Daemon.4522client # [ 98.953416] nix-daemon[1149]: accepted connection from pid 1146, user root (trusted)4523worker2 # [ 99.103695] nix-grpc-daemon[786]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:375944524worker1 # [ 99.094613] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544525worker1 # [ 99.109995] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04526worker1 # [ 99.118024] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14527worker1 # [ 99.127797] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544528worker1 # [ 99.139649] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544529worker1 # [ 99.154792] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395544530worker1 # [ 99.170169] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4531lb # [ 99.259541] envoy[895]: conn peer=[2001:db8:1::1]:45576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2976 tx=951 ms=64194532lb # [ 99.265029] envoy[895]: conn peer=[2001:db8:1::1]:35438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2986 tx=797 ms=43364533lb # [ 99.269923] envoy[895]: conn peer=[2001:db8:1::1]:35452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=50144534worker1 # [ 99.328779] nix-daemon[856]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)4535worker1 # [ 99.348450] nix-daemon[856]: reaped child process 1162, status = succeeded4536lb # [ 99.353412] niks3-server[985]: 2026/09/23 16:14:05 INFO Received push request method=POST path=/api/pushes4537worker1 # [ 99.365784] nix-grpc-daemon[870]: time=2026-09-23T16:14:05.703Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4538worker1 # [ 99.367227] nix-grpc-daemon[870]: time=2026-09-23T16:14:05.703Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4539lb # [ 99.388550] niks3-server[985]: 2026/09/23 16:14:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4540worker1 # [ 99.407565] nix-grpc-daemon[870]: time=2026-09-23T16:14:05.745Z level=INFO msg="Uploading 1 narinfos"4541lb # [ 99.405611] niks3-server[985]: 2026/09/23 16:14:05 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign4542lb # [ 99.413423] niks3-server[985]: 2026/09/23 16:14:05 INFO Signed narinfos id=21 count=14543lb # [ 99.421425] niks3-server[985]: 2026/09/23 16:14:05 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4544lb # [ 99.436224] niks3-server[985]: 2026/09/23 16:14:05 INFO Received complete push request method=POST path=/api/pushes/21/complete4545worker1 # [ 99.446935] nix-grpc-daemon[870]: time=2026-09-23T16:14:05.785Z level=INFO msg="Upload complete. (260ms)"4546lb # [ 99.446852] niks3-server[985]: 2026/09/23 16:14:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4547worker1 # [ 99.459128] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=5124548worker1 # [ 99.476468] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544549worker1 # [ 99.483965] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4550worker1 # [ 99.490892] nix-daemon[856]: reaped child process 1157, status = succeeded4551worker1 # [ 99.503724] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4552worker1 # [ 100.591587] nix-daemon[856]: reaped child process 1166, status = succeeded4553worker1 # [ 100.655946] nix-daemon[856]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)4554worker1 # [ 100.674294] nix-daemon[856]: reaped child process 1185, status = succeeded4555lb # [ 100.681723] niks3-server[985]: 2026/09/23 16:14:07 INFO Received push request method=POST path=/api/pushes4556worker1 # [ 100.696466] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.032Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4557worker1 # [ 100.698080] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.032Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4558lb # [ 100.729158] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4559worker1 # [ 100.742121] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.080Z level=INFO msg="Uploading 1 narinfos"4560lb # [ 100.742629] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4561lb # [ 100.753021] niks3-server[985]: 2026/09/23 16:14:07 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign4562lb # [ 100.764442] niks3-server[985]: 2026/09/23 16:14:07 INFO Signed narinfos id=22 count=14563lb # [ 100.766804] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4564lb # [ 100.775107] niks3-server[985]: 2026/09/23 16:14:07 INFO Received complete push request method=POST path=/api/pushes/22/complete4565worker1 # [ 100.777342] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.115Z level=INFO msg="Upload complete. (259ms)"4566lb # [ 100.779630] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4567worker1 # [ 100.792871] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14568worker2 # [ 100.811881] nix-grpc-daemon[786]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375944569worker1 # [ 100.809791] nix-daemon[856]: reaped child process 1164, status = succeeded4570worker2 # [ 100.825475] nix-grpc-daemon[786]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4571worker1 # [ 100.854429] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395544572worker1 # [ 100.862291] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=1284573worker1 # [ 101.101788] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:395544574worker2 # [ 101.120927] nix-grpc-daemon[786]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944575worker2 # [ 101.133264] nix-grpc-daemon[869]: time=2026-09-23T16:14:07.456Z level=INFO msg="All 1 paths already cached"4576worker1 # [ 101.131491] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04577worker1 # [ 101.138503] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14578worker1 # [ 101.147401] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544579worker1 # [ 101.158437] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544580worker1 # [ 101.172780] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.511Z level=INFO msg="All 1 paths already cached"4581worker1 # [ 101.182690] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395544582worker1 # [ 101.196949] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4583worker1 # [ 101.355716] nix-daemon[856]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)4584worker1 # [ 101.372003] nix-daemon[856]: reaped child process 1194, status = succeeded4585lb # [ 101.377739] niks3-server[985]: 2026/09/23 16:14:07 INFO Received push request method=POST path=/api/pushes4586worker1 # [ 101.388967] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.727Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4587worker1 # [ 101.390411] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.727Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4588lb # [ 101.408144] niks3-server[985]: 2026/09/23 16:14:07 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign4589worker1 # [ 101.418008] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.756Z level=INFO msg="Uploading 1 narinfos"4590lb # [ 101.421468] niks3-server[985]: 2026/09/23 16:14:07 INFO Signed narinfos id=23 count=14591lb # [ 101.430861] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4592worker1 # [ 101.446283] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.784Z level=INFO msg="Upload complete. (233ms)"4593lb # [ 101.445280] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4594lb # [ 101.450421] niks3-server[985]: 2026/09/23 16:14:07 INFO Received complete push request method=POST path=/api/pushes/23/complete4595worker1 # [ 101.456286] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=6244596lb # [ 101.455395] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4597worker1 # [ 101.471051] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544598worker1 # [ 101.477561] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4599worker1 # [ 101.482873] nix-daemon[856]: reaped child process 1189, status = succeeded4600worker1 # [ 101.494998] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4601worker1 # [ 102.587214] nix-daemon[856]: reaped child process 1198, status = succeeded4602worker1 # [ 102.670934] nix-daemon[856]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)4603worker1 # [ 102.691166] nix-daemon[856]: reaped child process 1217, status = succeeded4604lb # [ 102.693084] niks3-server[985]: 2026/09/23 16:14:09 INFO Received push request method=POST path=/api/pushes4605worker1 # [ 102.708438] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.044Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4606worker1 # [ 102.710173] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.044Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4607lb # [ 102.748484] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4608worker1 # [ 102.762788] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.101Z level=INFO msg="Uploading 1 narinfos"4609lb # [ 102.762496] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4610lb # [ 102.772863] niks3-server[985]: 2026/09/23 16:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign4611lb # [ 102.787910] niks3-server[985]: 2026/09/23 16:14:09 INFO Signed narinfos id=24 count=14612worker1 # [ 102.796742] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.135Z level=INFO msg="Upload complete. (289ms)"4613lb # [ 102.796380] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4614lb # [ 102.804426] niks3-server[985]: 2026/09/23 16:14:09 INFO Received complete push request method=POST path=/api/pushes/24/complete4615worker1 # [ 102.804582] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14616lb # [ 102.810179] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4617worker1 # [ 102.815011] nix-daemon[856]: reaped child process 1196, status = succeeded4618worker2 # [ 102.836119] nix-grpc-daemon[786]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375944619worker2 # [ 102.844283] nix-grpc-daemon[786]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4620worker1 # [ 102.868587] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395544621worker1 # [ 102.875733] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=1764622worker1 # [ 103.114876] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:395544623worker1 # [ 103.122598] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544624worker1 # [ 103.130329] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.466Z level=INFO msg="All 2 paths already cached"4625worker1 # [ 103.142281] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04626worker1 # [ 103.151171] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14627worker1 # [ 103.159851] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544628worker1 # [ 103.167417] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544629worker1 # [ 103.178469] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.517Z level=INFO msg="All 2 paths already cached"4630worker1 # [ 103.188566] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395544631worker1 # [ 103.202618] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4632worker1 # [ 103.349070] nix-daemon[856]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)4633worker1 # [ 103.371293] nix-daemon[856]: reaped child process 1226, status = succeeded4634lb # [ 103.373235] niks3-server[985]: 2026/09/23 16:14:09 INFO Received push request method=POST path=/api/pushes4635worker1 # [ 103.385165] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.723Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4636worker1 # [ 103.386679] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.723Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4637lb # [ 103.404673] niks3-server[985]: 2026/09/23 16:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign4638worker1 # [ 103.414952] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.753Z level=INFO msg="Uploading 1 narinfos"4639lb # [ 103.419640] niks3-server[985]: 2026/09/23 16:14:09 INFO Signed narinfos id=25 count=14640lb # [ 103.427491] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4641lb # [ 103.440842] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4642lb # [ 103.444559] niks3-server[985]: 2026/09/23 16:14:09 INFO Received complete push request method=POST path=/api/pushes/25/complete4643worker1 # [ 103.450346] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.788Z level=INFO msg="Upload complete. (232ms)"4644lb # [ 103.451520] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4645worker1 # [ 103.460370] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=7524646worker1 # [ 103.472603] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544647worker1 # [ 103.484229] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4648worker1 # [ 103.492647] nix-daemon[856]: reaped child process 1221, status = succeeded4649worker1 # [ 103.503132] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4650worker1 # [ 104.792776] nix-daemon[856]: reaped child process 1230, status = succeeded4651worker1 # [ 104.876950] nix-daemon[856]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)4652worker1 # [ 104.898121] nix-daemon[856]: reaped child process 1249, status = succeeded4653lb # [ 104.902804] niks3-server[985]: 2026/09/23 16:14:11 INFO Received push request method=POST path=/api/pushes4654worker1 # [ 104.916444] nix-grpc-daemon[870]: time=2026-09-23T16:14:11.253Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4655worker1 # [ 104.918049] nix-grpc-daemon[870]: time=2026-09-23T16:14:11.253Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4656lb # [ 104.947375] niks3-server[985]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4657worker1 # [ 104.962231] nix-grpc-daemon[870]: time=2026-09-23T16:14:11.300Z level=INFO msg="Uploading 1 narinfos"4658lb # [ 104.963686] niks3-server[985]: 2026/09/23 16:14:11 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign4659lb # [ 104.973918] niks3-server[985]: 2026/09/23 16:14:11 INFO Signed narinfos id=26 count=14660lb # [ 104.979513] niks3-server[985]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4661lb # [ 104.991311] niks3-server[985]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4662worker1 # [ 105.000490] nix-grpc-daemon[870]: time=2026-09-23T16:14:11.339Z level=INFO msg="Upload complete. (287ms)"4663lb # [ 104.999200] niks3-server[985]: 2026/09/23 16:14:11 INFO Received complete push request method=POST path=/api/pushes/26/complete4664lb # [ 105.002963] niks3-server[985]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4665worker1 # [ 105.017454] nix-grpc-daemon[487]: ts=2026-09-23T16:14:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14666worker1 # [ 105.035683] nix-grpc-daemon[487]: ts=2026-09-23T16:14:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395544667worker1 # [ 105.040312] nix-daemon[856]: reaped child process 1228, status = succeeded4668worker2 # [ 105.075387] nix-grpc-daemon[786]: ts=2026-09-23T16:14:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375944669worker2 # [ 105.095937] nix-grpc-daemon[786]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4670worker2 # [ 105.119825] nix-grpc-daemon[786]: ts=2026-09-23T16:14:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_out=2324671client # [ 105.230371] nix-daemon[1149]: reaped child process 1151, status = succeeded4672client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.55 seconds)4673client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4674client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4675client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4676client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4677client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4678client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4679(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.68 seconds)4680subtest: an input only one worker has still reaches the builder4681client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4682client # this derivation will be built:4683client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4684client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4685client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.44 seconds)4686client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4687client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.19 seconds)4688worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4689worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)4690worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4691worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4692client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24693client # [ 106.480455] nix-daemon[1149]: accepted connection from pid 1231, user root (trusted)4694client # this derivation will be built:4695client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4696client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4697worker2 # [ 106.643635] nix-grpc-daemon[786]: ts=2026-09-23T16:14:12Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:375944698worker2 # [ 106.652268] nix-grpc-daemon[786]: ts=2026-09-23T16:14:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944699client # copying 1 paths...4700client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4701worker1 # [ 106.650837] nix-grpc-daemon[487]: ts=2026-09-23T16:14:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395544702worker1 # [ 106.668236] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4703worker1 # [ 106.839486] nix-daemon[856]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)4704worker1 # [ 106.859269] nix-daemon[856]: reaped child process 1261, status = succeeded4705lb # [ 106.862933] niks3-server[985]: 2026/09/23 16:14:13 INFO Received push request method=POST path=/api/pushes4706worker1 # [ 106.877079] nix-grpc-daemon[870]: time=2026-09-23T16:14:13.214Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4707worker1 # [ 106.878784] nix-grpc-daemon[870]: time=2026-09-23T16:14:13.214Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4708lb # [ 106.896472] niks3-server[985]: 2026/09/23 16:14:13 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign4709worker1 # [ 106.908676] nix-grpc-daemon[870]: time=2026-09-23T16:14:13.246Z level=INFO msg="Uploading 1 narinfos"4710lb # [ 106.910117] niks3-server[985]: 2026/09/23 16:14:13 INFO Signed narinfos id=27 count=14711lb # [ 106.918443] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4712lb # [ 106.935838] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4713lb # [ 106.943364] niks3-server[985]: 2026/09/23 16:14:13 INFO Received complete push request method=POST path=/api/pushes/27/complete4714worker1 # [ 106.950411] nix-grpc-daemon[870]: time=2026-09-23T16:14:13.289Z level=INFO msg="Upload complete. (265ms)"4715lb # [ 106.951604] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4716worker1 # [ 106.960281] nix-grpc-daemon[487]: ts=2026-09-23T16:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=1204717worker1 # [ 106.968944] nix-daemon[856]: reaped child process 1256, status = succeeded4718worker1 # [ 106.977849] nix-grpc-daemon[487]: ts=2026-09-23T16:14:13Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04719worker1 # [ 106.982249] nix-grpc-daemon[487]: ts=2026-09-23T16:14:13Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14720worker2 # [ 107.000900] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944721worker2 # [ 107.016927] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944722client # copying 2 paths to 'lb:50051'...4723client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4724worker2 # [ 107.036240] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375944725client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4726worker2 # [ 107.059501] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4727worker2 # [ 107.232126] nix-daemon[844]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4728worker2 # [ 107.253726] nix-daemon[844]: reaped child process 1080, status = succeeded4729lb # [ 107.241633] niks3-server[985]: 2026/09/23 16:14:13 INFO Received push request method=POST path=/api/pushes4730worker2 # [ 107.271876] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.594Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4731worker2 # [ 107.275489] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.594Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4732worker2 # [ 107.277892] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.594Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4733lb # [ 107.285947] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4734lb # [ 107.303226] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4735lb # [ 107.312519] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4736worker2 # [ 107.337426] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.661Z level=INFO msg="Uploading 2 narinfos"4737lb # [ 107.321545] niks3-server[985]: 2026/09/23 16:14:13 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign4738lb # [ 107.328958] niks3-server[985]: 2026/09/23 16:14:13 INFO Signed narinfos id=28 count=24739lb # [ 107.334737] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4740lb # [ 107.347859] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4741lb # [ 107.360720] niks3-server[985]: 2026/09/23 16:14:13 INFO Received complete push request method=POST path=/api/pushes/28/complete4742worker2 # [ 107.386505] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.711Z level=INFO msg="Upload complete. (303ms)"4743lb # [ 107.371303] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4744worker2 # [ 107.396388] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=2 nar_bytes_in=9844745worker2 # [ 107.412952] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944746worker2 # [ 107.425949] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4747worker2 # [ 107.433208] nix-daemon[844]: reaped child process 1075, status = succeeded4748worker2 # [ 107.451243] nix-grpc-daemon[786]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4749client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4750client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4751worker2 # [ 107.473923] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4752client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4753worker2 # [ 108.740177] nix-daemon[844]: reaped child process 1085, status = succeeded4754worker2 # [ 108.849408] nix-daemon[844]: accepted connection from pid 1101, user nix-grpc-daemon (trusted)4755worker2 # [ 108.869548] nix-daemon[844]: reaped child process 1104, status = succeeded4756lb # [ 108.852974] niks3-server[985]: 2026/09/23 16:14:15 INFO Received push request method=POST path=/api/pushes4757worker2 # [ 108.888427] nix-grpc-daemon[869]: time=2026-09-23T16:14:15.210Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4758lb # [ 108.896587] niks3-server[985]: 2026/09/23 16:14:15 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4759worker2 # [ 108.925582] nix-grpc-daemon[869]: time=2026-09-23T16:14:15.250Z level=INFO msg="Uploading 1 narinfos"4760lb # [ 108.908974] niks3-server[985]: 2026/09/23 16:14:15 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign4761lb # [ 108.920082] niks3-server[985]: 2026/09/23 16:14:15 INFO Signed narinfos id=29 count=14762lb # [ 108.925507] niks3-server[985]: 2026/09/23 16:14:15 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4763lb # [ 108.944704] niks3-server[985]: 2026/09/23 16:14:15 INFO Received complete push request method=POST path=/api/pushes/29/complete4764lb # [ 108.951627] niks3-server[985]: 2026/09/23 16:14:15 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4765worker2 # [ 108.967157] nix-grpc-daemon[869]: time=2026-09-23T16:14:15.292Z level=INFO msg="Upload complete. (279ms)"4766client # copying 1 paths...4767worker2 # [ 108.981149] nix-grpc-daemon[786]: ts=2026-09-23T16:14:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=14768worker1 # [ 108.970977] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395544769worker2 # [ 108.999292] nix-daemon[844]: reaped child process 1082, status = succeeded4770worker1 # [ 108.986933] nix-grpc-daemon[487]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4771client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4772worker1 # [ 109.029045] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395544773worker1 # [ 109.036949] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=1204774client # [ 109.140244] nix-daemon[1149]: reaped child process 1234, status = succeeded4775client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.87 seconds)4776client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24777lb # [ 109.261968] envoy[895]: conn peer=[2001:db8:1::1]:42738 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=304778lb # [ 109.266985] envoy[895]: conn peer=[2001:db8:1::1]:35458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3433 tx=1942 ms=17924779lb # [ 109.271837] envoy[895]: conn peer=[2001:db8:1::1]:42754 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2156 ms=17914780lb # [ 109.276782] envoy[895]: conn peer=[2001:db8:1::1]:42768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=294781lb # [ 109.282287] envoy[895]: conn peer=[2001:db8:1::1]:42780 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=407 ms=754782lb # [ 109.287852] envoy[895]: conn peer=[2001:db8:1::1]:42778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4340 tx=2390 ms=20124783client # [ 109.309294] nix-daemon[1149]: accepted connection from pid 1253, user root (trusted)4784lb # [ 109.292851] envoy[895]: conn peer=[2001:db8:1::1]:42796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4240 tx=2073 ms=24344785lb # [ 109.297605] envoy[895]: conn peer=[2001:db8:1::1]:42810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=334786client # this derivation will be built:4787client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4788worker1 # [ 109.442680] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:395544789client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4790worker1 # [ 109.450643] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544791worker1 # [ 109.459249] nix-grpc-daemon[870]: time=2026-09-23T16:14:15.795Z level=INFO msg="All 1 paths already cached"4792worker1 # [ 109.471249] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04793worker1 # [ 109.479221] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14794worker1 # [ 109.488345] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544795worker1 # [ 109.499390] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395544796client # copying 1 paths to 'lb:50051'...4797client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4798worker1 # [ 109.520746] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395544799worker1 # [ 109.535810] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4800worker1 # [ 109.556214] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5)4801worker1 # [ 109.605277] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5)4802worker1 # [ 109.732342] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 361 ms (attempt 3/5)4803worker1 # [ 110.097065] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 376 ms (attempt 4/5)4804worker1 # [ 110.474578] nix-grpc-daemon[487]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4805worker1 # [ 110.481526] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4806worker1 # [ 110.490256] nix-grpc-daemon[487]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4807worker1 # [ 110.661636] nix-daemon[856]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)4808worker1 # [ 110.684495] nix-daemon[856]: reaped child process 1278, status = succeeded4809lb # [ 110.686791] niks3-server[985]: 2026/09/23 16:14:17 INFO Received push request method=POST path=/api/pushes4810worker1 # [ 110.699825] nix-grpc-daemon[870]: time=2026-09-23T16:14:17.037Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4811worker1 # [ 110.701571] nix-grpc-daemon[870]: time=2026-09-23T16:14:17.037Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4812lb # [ 110.716907] niks3-server[985]: 2026/09/23 16:14:17 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4813worker1 # [ 110.734685] nix-grpc-daemon[870]: time=2026-09-23T16:14:17.072Z level=INFO msg="Uploading 1 narinfos"4814lb # [ 110.732840] niks3-server[985]: 2026/09/23 16:14:17 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign4815lb # [ 110.740271] niks3-server[985]: 2026/09/23 16:14:17 INFO Signed narinfos id=30 count=14816lb # [ 110.749376] niks3-server[985]: 2026/09/23 16:14:17 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4817lb # [ 110.761440] niks3-server[985]: 2026/09/23 16:14:17 INFO Received complete push request method=POST path=/api/pushes/30/complete4818lb # [ 110.768776] niks3-server[985]: 2026/09/23 16:14:17 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4819worker1 # [ 110.771492] nix-grpc-daemon[870]: time=2026-09-23T16:14:17.110Z level=INFO msg="Upload complete. (260ms)"4820worker1 # [ 110.782784] nix-grpc-daemon[487]: ts=2026-09-23T16:14:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 paths=1 nar_bytes_in=5684821worker1 # [ 110.794565] nix-grpc-daemon[487]: ts=2026-09-23T16:14:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395544822worker1 # [ 110.807215] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4823worker1 # [ 110.813399] nix-daemon[856]: reaped child process 1268, status = succeeded4824client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4825worker1 # [ 110.828604] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4826client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4827client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4828worker1 # [ 111.878412] nix-daemon[856]: reaped child process 1282, status = succeeded4829worker1 # [ 111.973816] nix-daemon[856]: accepted connection from pid 1298, user nix-grpc-daemon (trusted)4830worker1 # [ 111.996568] nix-daemon[856]: reaped child process 1301, status = succeeded4831lb # [ 111.999329] niks3-server[985]: 2026/09/23 16:14:18 INFO Received push request method=POST path=/api/pushes4832worker1 # [ 112.012405] nix-grpc-daemon[870]: time=2026-09-23T16:14:18.349Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4833lb # [ 112.033995] niks3-server[985]: 2026/09/23 16:14:18 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4834worker1 # [ 112.047905] nix-grpc-daemon[870]: time=2026-09-23T16:14:18.386Z level=INFO msg="Uploading 1 narinfos"4835lb # [ 112.046672] niks3-server[985]: 2026/09/23 16:14:18 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign4836lb # [ 112.056328] niks3-server[985]: 2026/09/23 16:14:18 INFO Signed narinfos id=31 count=14837lb # [ 112.067057] niks3-server[985]: 2026/09/23 16:14:18 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4838worker1 # [ 112.077548] nix-grpc-daemon[870]: time=2026-09-23T16:14:18.416Z level=INFO msg="Upload complete. (237ms)"4839lb # [ 112.076798] niks3-server[985]: 2026/09/23 16:14:18 INFO Received complete push request method=POST path=/api/pushes/31/complete4840lb # [ 112.083007] niks3-server[985]: 2026/09/23 16:14:18 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4841worker1 # [ 112.089025] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=14842client # copying 1 paths...4843worker2 # [ 112.119801] nix-grpc-daemon[786]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375944844worker1 # [ 112.107077] nix-daemon[856]: reaped child process 1280, status = succeeded4845worker2 # [ 112.128529] nix-grpc-daemon[786]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4846client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4847worker1 # [ 112.159368] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395544848worker1 # [ 112.167291] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=1204849client # [ 112.278695] nix-daemon[1149]: reaped child process 1256, status = succeeded4850client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.14 seconds)4851client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24852client # [ 112.451344] nix-daemon[1149]: accepted connection from pid 1275, user root (trusted)4853client # this derivation will be built:4854client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4855worker1 # [ 112.601624] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:395544856client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4857worker2 # [ 112.624864] nix-grpc-daemon[786]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944858worker2 # [ 112.635739] nix-grpc-daemon[869]: time=2026-09-23T16:14:18.960Z level=INFO msg="All 1 paths already cached"4859worker1 # [ 112.638517] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04860worker1 # [ 112.645409] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14861worker2 # [ 112.663824] nix-grpc-daemon[786]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944862worker2 # [ 112.680585] nix-grpc-daemon[786]: ts=2026-09-23T16:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375944863worker2 # [ 112.696738] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.021Z level=INFO msg="All 1 paths already cached"4864client # copying 1 paths to 'lb:50051'...4865client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4866worker2 # [ 112.708437] nix-grpc-daemon[786]: ts=2026-09-23T16:14:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375944867worker2 # [ 112.724804] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4868worker2 # [ 112.884759] nix-daemon[844]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4869worker2 # [ 112.905288] nix-daemon[844]: reaped child process 1112, status = succeeded4870lb # [ 112.896654] niks3-server[985]: 2026/09/23 16:14:19 INFO Received push request method=POST path=/api/pushes4871worker2 # [ 112.924659] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.249Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4872worker2 # [ 112.926606] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.249Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4873lb # [ 112.939379] niks3-server[985]: 2026/09/23 16:14:19 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4874worker2 # [ 112.970657] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.295Z level=INFO msg="Uploading 1 narinfos"4875lb # [ 112.957751] niks3-server[985]: 2026/09/23 16:14:19 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign4876lb # [ 112.967066] niks3-server[985]: 2026/09/23 16:14:19 INFO Signed narinfos id=32 count=14877lb # [ 112.973196] niks3-server[985]: 2026/09/23 16:14:19 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4878lb # [ 112.984396] niks3-server[985]: 2026/09/23 16:14:19 INFO Received complete push request method=POST path=/api/pushes/32/complete4879worker2 # [ 113.012717] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.337Z level=INFO msg="Upload complete. (270ms)"4880lb # [ 112.997151] niks3-server[985]: 2026/09/23 16:14:19 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4881worker2 # [ 113.025491] nix-grpc-daemon[786]: ts=2026-09-23T16:14:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=5684882worker2 # [ 113.040693] nix-grpc-daemon[786]: ts=2026-09-23T16:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375944883worker2 # [ 113.051799] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4884worker2 # [ 113.058638] nix-daemon[844]: reaped child process 1107, status = succeeded4885worker2 # [ 113.070661] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4886client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4887client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4888client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4889worker2 # [ 113.941912] nix-daemon[844]: reaped child process 1116, status = succeeded4890worker2 # [ 114.061357] nix-daemon[844]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)4891worker2 # [ 114.082257] nix-daemon[844]: reaped child process 1135, status = succeeded4892lb # [ 114.070494] niks3-server[985]: 2026/09/23 16:14:20 INFO Received push request method=POST path=/api/pushes4893worker2 # [ 114.096527] nix-grpc-daemon[869]: time=2026-09-23T16:14:20.421Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4894lb # [ 114.106971] niks3-server[985]: 2026/09/23 16:14:20 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4895worker2 # [ 114.135225] nix-grpc-daemon[869]: time=2026-09-23T16:14:20.460Z level=INFO msg="Uploading 1 narinfos"4896lb # [ 114.122983] niks3-server[985]: 2026/09/23 16:14:20 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign4897lb # [ 114.132213] niks3-server[985]: 2026/09/23 16:14:20 INFO Signed narinfos id=33 count=14898lb # [ 114.141196] niks3-server[985]: 2026/09/23 16:14:20 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4899worker2 # [ 114.167812] nix-grpc-daemon[869]: time=2026-09-23T16:14:20.492Z level=INFO msg="Upload complete. (281ms)"4900lb # [ 114.152351] niks3-server[985]: 2026/09/23 16:14:20 INFO Received complete push request method=POST path=/api/pushes/33/complete4901lb # [ 114.157404] niks3-server[985]: 2026/09/23 16:14:20 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4902client # copying 1 paths...4903worker2 # [ 114.181218] nix-grpc-daemon[786]: ts=2026-09-23T16:14:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=14904worker1 # [ 114.178120] nix-grpc-daemon[487]: ts=2026-09-23T16:14:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395544905worker2 # [ 114.200558] nix-daemon[844]: reaped child process 1114, status = succeeded4906worker1 # [ 114.186373] nix-grpc-daemon[487]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4907client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4908worker1 # [ 114.230156] nix-grpc-daemon[487]: ts=2026-09-23T16:14:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395544909worker1 # [ 114.237153] nix-grpc-daemon[487]: ts=2026-09-23T16:14:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=1204910client # [ 114.339796] nix-daemon[1149]: reaped child process 1278, status = succeeded4911client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.06 seconds)4912(finished: subtest: an input only one worker has still reaches the builder, in 8.97 seconds)4913subtest: upload whose reference the worker lacks is completed from the cache4914client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4915client # this derivation will be built:4916client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4917client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4918client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.34 seconds)4919client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4920client # this derivation will be built:4921client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4922client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4923client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.30 seconds)4924client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4925client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)4926worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4927worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)4928worker2 # [ 115.367908] systemd[1]: Stopping Nix worker-protocol over gRPC...4929worker2 # [ 115.859273] nix-grpc-daemon[786]: ts=2026-09-23T16:14:22Z level=info event=signal_exit4930worker2 # [ 117.839224] nix-grpc-daemon[786]: ts=2026-09-23T16:14:24Z level=info event=scheduler_restarting4931worker1 # [ 117.828670] nix-grpc-daemon[487]: ts=2026-09-23T16:14:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590944932worker1 # [ 117.829112] nix-grpc-daemon[487]: ts=2026-09-23T16:14:24Z level=info event=worker_gone addr=192.168.1.4:500514933worker2 # [ 117.869478] nix-grpc-daemon[786]: ts=2026-09-23T16:14:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4934worker2 # [ 117.904167] nix-daemon[844]: reaped child process 846, status = succeeded4935worker2 # [ 118.442077] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4936worker2 # [ 118.449374] systemd[1]: Stopped Nix worker-protocol over gRPC.4937worker2 # [ 118.455404] systemd[1]: nix-grpc-daemon.service: Consumed 2.599s CPU time over 1min 22.813s wall clock time, 63.4M memory peak, 154.9K incoming IP traffic, 202.8K outgoing IP traffic.4938lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux4939lb: must succeed: curl -sf localhost:9901/clusters4940lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4941lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.07 seconds)4942client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24943worker1 # [ 118.739207] nix-grpc-daemon[487]: ts=2026-09-23T16:14:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395544944worker1 # [ 118.749155] nix-grpc-daemon[487]: ts=2026-09-23T16:14:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395544945client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4946client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.26 seconds)4947worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4948worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)4949worker2 # [ 118.893715] systemd[1]: Starting scheduler-token.service...4950worker2 # [ 118.953859] systemd[1]: scheduler-token.service: Deactivated successfully.4951worker2 # [ 118.960281] systemd[1]: Finished scheduler-token.service.4952worker2 # [ 118.961735] systemd[1]: scheduler-token.service: Consumed 33ms CPU time over 62ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4953worker2 # [ 118.978167] systemd[1]: Starting Nix worker-protocol over gRPC...4954worker2 # [ 119.082116] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4955worker2 # [ 119.084470] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4956worker2 # [ 119.088337] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14957worker2 # [ 119.094844] nix-grpc-daemon[1159]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4958worker2 # [ 119.104636] nix-grpc-daemon[1159]: E0000 00:00:1790180065.417813 1159 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4959worker2 # [ 119.120137] nix-grpc-daemon[1159]: E0000 00:00:1790180065.419839 1159 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4960worker2 # [ 119.122422] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket4961worker2 # [ 119.143857] nix-grpc-daemon[1159]: E0000 00:00:1790180065.434244 1199 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4962worker2 # [ 119.147018] systemd[1]: Started Nix worker-protocol over gRPC.4963worker2 # [ 119.148040] nix-grpc-daemon[1159]: E0000 00:00:1790180065.453502 1199 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4964worker2 # [ 119.150287] nix-grpc-daemon[1159]: E0000 00:00:1790180065.453689 1199 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4965worker2 # [ 119.155927] nix-grpc-daemon[1159]: E0000 00:00:1790180065.468055 1199 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4966worker2 # [ 119.160470] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=scheduler_connected addr=lb:500514967worker1 # [ 119.151312] nix-grpc-daemon[487]: ts=2026-09-23T16:14:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:590944968worker1 # [ 119.176609] nix-grpc-daemon[487]: ts=2026-09-23T16:14:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04969worker1 # [ 119.182415] systemd[1]: Stopping Nix worker-protocol over gRPC...4970lb # [ 119.261572] envoy[895]: conn peer=[2001:db8:1::1]:42826 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3757 tx=2125 ms=27434971lb # [ 119.266632] envoy[895]: conn peer=[2001:db8:1::1]:43390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=334972lb # [ 119.271471] envoy[895]: conn peer=[2001:db8:1::1]:43396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2126 ms=16544973lb # [ 119.276738] envoy[895]: conn peer=[2001:db8:1::1]:43404 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=294974lb # [ 119.281849] envoy[895]: conn peer=[2001:db8:1::4]:37578 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8622 tx=1175 ms=819734975lb # [ 119.287334] envoy[895]: conn peer=[2001:db8:1::1]:43410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=334976worker1 # [ 119.826150] nix-grpc-daemon[487]: ts=2026-09-23T16:14:26Z level=info event=signal_exit4977lb # [ 120.568867] niks3-server[985]: 2026/09/23 16:14:26 INFO lead: released remote=192.168.1.3:540784978worker2 # [ 121.588496] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:27Z level=info event=scheduler_restarting addr=lb:500514979lb # [ 121.573397] envoy[895]: [2026-09-23 16:14:27.895][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}4980worker1 # [ 121.582375] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=scheduler_restarting4981worker1 # [ 121.582767] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590944982worker1 # [ 121.582891] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=worker_gone addr=192.168.1.4:500514983worker1 # [ 121.594994] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=scheduler_restarting addr=lb:500514984worker1 # [ 121.595242] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590944985worker1 # [ 121.595358] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=worker_gone addr=192.168.1.3:500514986worker1 # [ 121.622479] nix-daemon[856]: reaped child process 858, status = succeeded4987worker2 # [ 121.788533] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:500514988worker2 # [ 121.794242] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4989worker1 # [ 121.804588] nix-grpc-daemon[487]: ts=2026-09-23T16:14:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4990worker2 # [ 121.995596] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:500514991worker2 # [ 122.001252] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4992worker2 # [ 122.201986] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:500514993worker2 # [ 122.207452] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4994worker2 # [ 122.408772] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:500514995worker1 # [ 122.397889] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4996worker2 # [ 122.414274] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4997worker1 # [ 122.408804] systemd[1]: Stopped Nix worker-protocol over gRPC.4998worker1 # [ 122.411440] systemd[1]: nix-grpc-daemon.service: Consumed 3.823s CPU time over 1min 51.958s wall clock time, 67.8M memory peak, 463.1K incoming IP traffic, 322.1K outgoing IP traffic.4999lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5000lb: must succeed: curl -sf localhost:9901/clusters5001lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5002lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.08 seconds)5003client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25004worker2 # [ 122.616990] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:500515005worker2 # [ 122.621870] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5006worker2 # [ 122.749538] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436425007worker2 # [ 122.765111] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5008worker2 # [ 122.792587] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5)5009worker2 # [ 122.822901] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:500515010worker2 # [ 122.828207] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5011worker2 # [ 122.898270] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 2/5)5012worker2 # [ 122.993528] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 267 ms (attempt 3/5)5013worker2 # [ 123.028901] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:500515014worker2 # [ 123.031647] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5015worker2 # [ 123.232919] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:500515016worker2 # [ 123.238235] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5017worker2 # [ 123.262640] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 545 ms (attempt 4/5)5018worker2 # [ 123.439564] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:500515019worker2 # [ 123.445235] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5020worker2 # [ 123.648396] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:500515021worker2 # [ 123.652584] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5022worker2 # [ 123.810355] nix-grpc-daemon[1159]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5023worker2 # [ 123.817405] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5024client # copying 2 paths...5025client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5026worker2 # [ 123.841462] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:436425027client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5028worker2 # [ 123.848885] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:500515029worker2 # [ 123.862752] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5030worker2 # [ 123.877291] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5031worker2 # [ 123.902952] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=niks3_push_started pid=12195032worker2 # [ 124.050881] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.375Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5033worker2 # [ 124.057906] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:500515034worker2 # [ 124.059997] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5035worker2 # [ 124.105527] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_take_over5036lb # [ 124.090109] niks3-server[985]: 2026/09/23 16:14:30 INFO lead: acquired remote=192.168.1.4:580125037worker2 # [ 124.169747] nix-daemon[844]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5038worker2 # [ 124.192181] nix-daemon[844]: reaped child process 1235, status = succeeded5039lb # [ 124.175841] niks3-server[985]: 2026/09/23 16:14:30 INFO Received push request method=POST path=/api/pushes5040worker2 # [ 124.223415] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.548Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5041worker2 # [ 124.229336] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.550Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5042worker2 # [ 124.231482] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.550Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5043worker2 # [ 124.258791] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:500515044worker2 # [ 124.261213] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045lb # [ 124.266532] niks3-server[985]: 2026/09/23 16:14:30 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5046lb # [ 124.282084] niks3-server[985]: 2026/09/23 16:14:30 INFO Signed narinfos id=34 count=25047worker2 # [ 124.300107] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.623Z level=INFO msg="Uploading 2 narinfos"5048lb # [ 124.293862] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5049lb # [ 124.306578] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5050lb # [ 124.321364] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5051lb # [ 124.334036] niks3-server[985]: 2026/09/23 16:14:30 INFO Received complete push request method=POST path=/api/pushes/34/complete5052lb # [ 124.346171] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5053lb # [ 124.358795] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5054lb # [ 124.378628] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5055worker2 # [ 124.407973] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.732Z level=INFO msg="Upload complete. (352ms)"5056worker2 # [ 124.420159] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=0 paths=2 nar_bytes_in=3125057worker2 # [ 124.437891] nix-daemon[844]: reaped child process 1217, status = succeeded5058client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.94 seconds)5059worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5060worker2 # [ 124.463965] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:500515061worker2 # [ 124.472333] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5062worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)5063worker1 # [ 124.564719] systemd[1]: Starting Nix worker-protocol over gRPC...5064worker2 # [ 124.671891] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:500515065worker2 # [ 124.688127] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=oidc_jwks provider=mock keys=15066worker2 # [ 124.693762] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:436485067worker2 # [ 124.699331] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05068worker1 # [ 124.692473] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5069worker1 # [ 124.694453] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5070worker1 # [ 124.698400] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15071worker1 # [ 124.704071] nix-grpc-daemon[1327]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5072worker1 # [ 124.705532] nix-grpc-daemon[1327]: E0000 00:00:1790180071.041692 1327 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5073worker1 # [ 124.709571] nix-grpc-daemon[1327]: E0000 00:00:1790180071.048105 1327 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5074worker1 # [ 124.713107] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5075worker1 # [ 124.714850] systemd[1]: Started Nix worker-protocol over gRPC.5076worker1 # [ 124.748724] nix-grpc-daemon[1327]: E0000 00:00:1790180071.060406 1370 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5077worker1 # [ 124.751135] nix-grpc-daemon[1327]: E0000 00:00:1790180071.077726 1370 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5078lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5079lb: must succeed: curl -sf localhost:9901/clusters5080worker1 # [ 124.761783] nix-grpc-daemon[1327]: E0000 00:00:1790180071.078750 1370 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5081worker1 # [ 124.764822] nix-grpc-daemon[1327]: E0000 00:00:1790180071.099547 1370 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5082worker1 # [ 124.779867] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=scheduler_connected addr=lb:500515083worker2 # [ 124.795799] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:436485084worker2 # [ 124.801155] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05085lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5086lb: must succeed: curl -sf localhost:9901/clusters5087lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5088lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.14 seconds)5089(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.56 seconds)5090subtest: a niks3 restart does not cost the leader its role5091lb: must succeed: curl -sf localhost:9901/clusters5092lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5093lb: must succeed: curl -sf localhost:9901/clusters5094lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5095lb # [ 126.015924] niks3-server[985]: 2026/09/23 16:14:32 INFO Shutdown signal received, draining in-flight requests timeout=10s5096lb # [ 126.023983] systemd[1]: Stopping niks3 server...5097lb # [ 126.025920] niks3-server[985]: 2026/09/23 16:14:32 INFO lead: released remote=192.168.1.4:580125098worker2 # [ 126.041537] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:32Z level=info event=scheduler_lock_lost status=2005099worker1 # [ 126.027382] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:32Z level=info event=scheduler_lock_lost status=2005100lb # [ 126.047920] systemd[1]: niks3.service: Deactivated successfully.5101lb # [ 126.049976] systemd[1]: Stopped niks3 server.5102lb # [ 126.051732] systemd[1]: niks3.service: Consumed 705ms CPU time over 1min 39.782s wall clock time, 27.7M memory peak, 98.2K incoming IP traffic, 140.7K outgoing IP traffic.5103lb # [ 126.062213] systemd[1]: Starting niks3 server...5104lb # [ 126.126983] niks3-server[1178]: 2026/09/23 16:14:32 goose: no migrations to run. current version: 202609231200005105lb # [ 126.135867] niks3-server[1178]: 2026/09/23 16:14:32 OK 1_commit_pending_closure.sql (7.09ms)5106lb # [ 126.139514] niks3-server[1178]: 2026/09/23 16:14:32 OK 2_object_stats_trigger.sql (2.3ms)5107lb # [ 126.142825] niks3-server[1178]: 2026/09/23 16:14:32 OK 3_commit_push.sql (2.91ms)5108lb # [ 126.143880] niks3-server[1178]: 2026/09/23 16:14:32 goose: up to current file version: 35109lb # [ 126.145787] niks3-server[1178]: 2026/09/23 16:14:32 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5110lb # [ 126.157257] niks3-server[1178]: 2026/09/23 16:14:32 INFO Read proxy enabled — serving cache objects from S35111lb # [ 126.158582] niks3-server[1178]: 2026/09/23 16:14:32 INFO Using socket-activated listener address=0.0.0.0:57515112lb # [ 126.159910] systemd[1]: Started niks3 server.5113lb # [ 126.163925] niks3-server[1178]: 2026/09/23 16:14:32 INFO systemd watchdog enabled interval=15s5114lb # [ 126.165728] niks3-server[1178]: 2026/09/23 16:14:32 INFO Starting HTTP server address=0.0.0.0:57515115lb: waiting for unit niks3.service5116lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5117lb: must succeed: curl -sf localhost:9901/clusters5118lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5119worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5120worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5121lb: must succeed: curl -sf localhost:9901/clusters5122lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5123lb: must succeed: curl -sf localhost:9901/clusters5124lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5125worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5126worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5127client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25128worker1 # [ 126.648170] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556685129worker1 # [ 126.663990] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5130client # don't know how to build these paths:5131client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5132worker1 # [ 126.690416] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556685133worker1 # [ 126.700770] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)5134worker1 # [ 126.763324] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)5135worker1 # [ 126.801517] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5)5136worker1 # [ 126.927771] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 538 ms (attempt 4/5)5137lb # [ 127.031117] niks3-server[1178]: 2026/09/23 16:14:33 INFO lead: acquired remote=192.168.1.4:469305138worker1 # [ 127.469681] nix-grpc-daemon[1327]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5139worker1 # [ 127.477270] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5140worker2 # [ 127.546105] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05141worker2 # [ 127.546492] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15142worker2 # [ 127.560465] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436425143worker2 # [ 127.580658] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436425144client # copying 1 paths to 'lb:50051'...5145client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5146worker2 # [ 127.597438] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:436425147worker2 # [ 127.612855] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5148worker2 # [ 127.768142] nix-daemon[844]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)5149worker2 # [ 127.786357] nix-daemon[844]: reaped child process 1257, status = succeeded5150lb # [ 127.779207] niks3-server[1178]: 2026/09/23 16:14:34 INFO Received push request method=POST path=/api/pushes5151worker2 # [ 127.811580] nix-grpc-daemon[1219]: time=2026-09-23T16:14:34.135Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5152worker2 # [ 127.813566] nix-grpc-daemon[1219]: time=2026-09-23T16:14:34.135Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5153lb # [ 127.832548] niks3-server[1178]: 2026/09/23 16:14:34 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5154lb # [ 127.845285] niks3-server[1178]: 2026/09/23 16:14:34 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5155lb # [ 127.872218] niks3-server[1178]: 2026/09/23 16:14:34 INFO Signed narinfos id=35 count=15156worker2 # [ 127.892117] nix-grpc-daemon[1219]: time=2026-09-23T16:14:34.216Z level=INFO msg="Uploading 1 narinfos"5157lb # [ 127.880157] niks3-server[1178]: 2026/09/23 16:14:34 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5158lb # [ 127.892276] niks3-server[1178]: 2026/09/23 16:14:34 INFO Received complete push request method=POST path=/api/pushes/35/complete5159worker2 # [ 127.917525] nix-grpc-daemon[1219]: time=2026-09-23T16:14:34.242Z level=INFO msg="Upload complete. (289ms)"5160lb # [ 127.904070] niks3-server[1178]: 2026/09/23 16:14:34 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5161worker2 # [ 127.929515] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=0 paths=1 nar_bytes_in=5445162worker2 # [ 127.944356] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436425163worker2 # [ 127.956260] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5164worker2 # [ 127.962675] nix-daemon[844]: reaped child process 1252, status = succeeded5165client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5166worker2 # [ 127.975697] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5167client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5168client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5169client # farm-a> LOG-farm-a-after-niks3-restart5170worker2 # [ 129.032491] nix-daemon[844]: reaped child process 1261, status = succeeded5171worker2 # [ 129.151717] nix-daemon[844]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)5172worker2 # [ 129.171269] nix-daemon[844]: reaped child process 1280, status = succeeded5173lb # [ 129.162495] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received push request method=POST path=/api/pushes5174worker2 # [ 129.199911] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.521Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5175worker2 # [ 129.202192] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.522Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5176lb # [ 129.220951] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5177worker2 # [ 129.249952] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.574Z level=INFO msg="Uploading 1 narinfos"5178lb # [ 129.234123] niks3-server[1178]: 2026/09/23 16:14:35 INFO Signed narinfos id=36 count=15179lb # [ 129.245904] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5180lb # [ 129.256518] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5181lb # [ 129.265897] envoy[895]: conn peer=[2001:db8:1::3]:40604 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2864 tx=1357 ms=852665182lb # [ 129.279386] envoy[895]: conn peer=[2001:db8:1::1]:52662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1010 tx=305 ms=16885183lb # [ 129.286235] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5184lb # [ 129.292333] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received complete push request method=POST path=/api/pushes/36/complete5185lb # [ 129.296165] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5186worker2 # [ 129.325546] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.650Z level=INFO msg="Upload complete. (336ms)"5187worker2 # [ 129.351164] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15188worker2 # [ 129.372236] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05189worker2 # [ 129.376286] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15190worker2 # [ 129.382459] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436425191worker2 # [ 129.387736] nix-daemon[844]: reaped child process 1259, status = succeeded5192worker2 # [ 129.391282] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436425193worker2 # [ 129.395869] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.720Z level=INFO msg="All 1 paths already cached"5194client # copying 1 paths to 'lb:50051'...5195client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5196worker2 # [ 129.405457] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:436425197worker2 # [ 129.417755] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5198worker2 # [ 129.561323] nix-daemon[844]: accepted connection from pid 1286, user nix-grpc-daemon (trusted)5199worker2 # [ 129.582529] nix-daemon[844]: reaped child process 1289, status = succeeded5200lb # [ 129.572142] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received push request method=POST path=/api/pushes5201worker2 # [ 129.598839] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.923Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5202worker2 # [ 129.601012] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.925Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5203worker2 # [ 129.640653] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.965Z level=INFO msg="Uploading 1 narinfos"5204lb # [ 129.614706] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5205lb # [ 129.629068] niks3-server[1178]: 2026/09/23 16:14:35 INFO Signed narinfos id=37 count=15206lb # [ 129.636461] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5207lb # [ 129.643867] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5208lb # [ 129.656046] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received complete push request method=POST path=/api/pushes/37/complete5209lb # [ 129.666538] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5210worker2 # [ 129.682885] nix-grpc-daemon[1219]: time=2026-09-23T16:14:36.007Z level=INFO msg="Upload complete. (251ms)"5211worker2 # [ 129.693313] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=0 paths=1 nar_bytes_in=6565212worker2 # [ 129.708600] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436425213worker2 # [ 129.719502] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5214worker2 # [ 129.728316] nix-daemon[844]: reaped child process 1284, status = succeeded5215client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5216worker2 # [ 129.737970] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5217client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5218worker2 # [ 129.983259] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:36Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:436585219client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5220client # farm-b> LOG-farm-b-after-niks3-restart5221worker2 # [ 130.646231] nix-daemon[844]: reaped child process 1293, status = succeeded5222worker2 # [ 130.718602] nix-daemon[844]: accepted connection from pid 1309, user nix-grpc-daemon (trusted)5223worker2 # [ 130.737794] nix-daemon[844]: reaped child process 1312, status = succeeded5224lb # [ 130.725985] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received push request method=POST path=/api/pushes5225worker2 # [ 130.753966] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.077Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5226worker2 # [ 130.755886] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.077Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5227lb # [ 130.773761] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5228worker2 # [ 130.798866] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.123Z level=INFO msg="Uploading 1 narinfos"5229lb # [ 130.787668] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5230lb # [ 130.796274] niks3-server[1178]: 2026/09/23 16:14:37 INFO Signed narinfos id=38 count=15231lb # [ 130.798378] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5232lb # [ 130.812436] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5233lb # [ 130.816786] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received complete push request method=POST path=/api/pushes/38/complete5234lb # [ 130.826566] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5235worker2 # [ 130.849531] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.174Z level=INFO msg="Upload complete. (297ms)"5236worker2 # [ 130.870534] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15237worker2 # [ 130.891269] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05238worker2 # [ 130.895356] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15239worker2 # [ 130.901393] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436425240worker2 # [ 130.906683] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436425241worker2 # [ 130.911873] nix-daemon[844]: reaped child process 1291, status = succeeded5242worker2 # [ 130.916225] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.241Z level=INFO msg="All 2 paths already cached"5243client # copying 1 paths to 'lb:50051'...5244client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5245worker2 # [ 130.925971] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:436425246worker2 # [ 130.940378] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5247worker2 # [ 131.078971] nix-daemon[844]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5248worker2 # [ 131.098178] nix-daemon[844]: reaped child process 1321, status = succeeded5249lb # [ 131.085334] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received push request method=POST path=/api/pushes5250worker2 # [ 131.111877] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.436Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5251worker2 # [ 131.113941] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.436Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5252lb # [ 131.111812] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5253lb # [ 131.123643] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5254worker2 # [ 131.148562] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.473Z level=INFO msg="Uploading 1 narinfos"5255lb # [ 131.134649] niks3-server[1178]: 2026/09/23 16:14:37 INFO Signed narinfos id=39 count=15256lb # [ 131.139163] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5257lb # [ 131.149194] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received complete push request method=POST path=/api/pushes/39/complete5258worker2 # [ 131.175140] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.500Z level=INFO msg="Upload complete. (224ms)"5259lb # [ 131.160470] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5260worker2 # [ 131.184264] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=0 paths=1 nar_bytes_in=7845261worker2 # [ 131.199299] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436425262worker2 # [ 131.205896] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5263worker2 # [ 131.210789] nix-daemon[844]: reaped child process 1316, status = succeeded5264worker2 # [ 131.222628] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5265client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5266client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5267client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5268client # farm-top> LOG-farm-top-after-niks3-restart5269worker2 # [ 132.487298] nix-daemon[844]: reaped child process 1325, status = succeeded5270worker2 # [ 132.620986] nix-daemon[844]: accepted connection from pid 1341, user nix-grpc-daemon (trusted)5271worker2 # [ 132.642425] nix-daemon[844]: reaped child process 1344, status = succeeded5272lb # [ 132.630606] niks3-server[1178]: 2026/09/23 16:14:38 INFO Received push request method=POST path=/api/pushes5273worker2 # [ 132.659006] nix-grpc-daemon[1219]: time=2026-09-23T16:14:38.982Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5274worker2 # [ 132.661067] nix-grpc-daemon[1219]: time=2026-09-23T16:14:38.982Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5275lb # [ 132.675552] niks3-server[1178]: 2026/09/23 16:14:39 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5276worker2 # [ 132.704802] nix-grpc-daemon[1219]: time=2026-09-23T16:14:39.029Z level=INFO msg="Uploading 1 narinfos"5277lb # [ 132.690478] niks3-server[1178]: 2026/09/23 16:14:39 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5278lb # [ 132.698540] niks3-server[1178]: 2026/09/23 16:14:39 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5279lb # [ 132.702656] niks3-server[1178]: 2026/09/23 16:14:39 INFO Signed narinfos id=40 count=15280lb # [ 132.711742] niks3-server[1178]: 2026/09/23 16:14:39 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5281lb # [ 132.719385] niks3-server[1178]: 2026/09/23 16:14:39 INFO Received complete push request method=POST path=/api/pushes/40/complete5282lb # [ 132.723707] niks3-server[1178]: 2026/09/23 16:14:39 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5283worker2 # [ 132.736656] nix-grpc-daemon[1219]: time=2026-09-23T16:14:39.061Z level=INFO msg="Upload complete. (302ms)"5284worker2 # [ 132.749361] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15285worker2 # [ 132.769835] nix-daemon[844]: reaped child process 1323, status = succeeded5286client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.36 seconds)5287client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5288client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds)5289(finished: subtest: a niks3 restart does not cost the leader its role, in 7.07 seconds)5290subtest: scheduler down: the other node takes the lock and keeps it5291lb: must succeed: curl -sf localhost:9901/clusters5292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5293lb: must succeed: curl -sf localhost:9901/clusters5294lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5295lb: must succeed: curl -sf localhost:9901/clusters5296lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5297lb: must succeed: curl -sf localhost:9901/clusters5298lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5299worker2 # [ 133.231196] systemd[1]: Stopping Nix worker-protocol over gRPC...5300worker2 # [ 134.109874] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:40Z level=info event=signal_exit5301lb # [ 135.043914] niks3-server[1178]: 2026/09/23 16:14:41 INFO lead: released remote=192.168.1.4:469305302worker1 # [ 136.042265] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_restarting addr=lb:500515303worker2 # [ 136.060521] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=scheduler_restarting5304lb # [ 136.041114] envoy[895]: [2026-09-23 16:14:42.363][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5305worker2 # [ 136.067676] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=scheduler_restarting addr=lb:500515306worker2 # [ 136.072426] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:436485307worker2 # [ 136.077297] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=worker_gone addr=192.168.1.3:500515308worker2 # [ 136.083087] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:436485309worker2 # [ 136.088149] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=worker_gone addr=192.168.1.4:500515310worker2 # [ 136.110797] nix-daemon[844]: reaped child process 1211, status = succeeded5311worker1 # [ 136.241983] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_connected addr=lb:500515312worker1 # [ 136.248084] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker2 # [ 136.278441] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5314worker2 # [ 136.343872] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5315worker2 # [ 136.350999] systemd[1]: Stopped Nix worker-protocol over gRPC.5316worker2 # [ 136.358554] systemd[1]: nix-grpc-daemon.service: Consumed 1.449s CPU time over 17.380s wall clock time, 39.6M memory peak, 201.9K incoming IP traffic, 125.9K outgoing IP traffic.5317worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5318worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)5319worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5320worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5321worker1 # [ 136.448539] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_connected addr=lb:500515322worker1 # [ 136.451207] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker1 # [ 136.652398] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_connected addr=lb:500515324worker1 # [ 136.658028] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5325worker1 # [ 136.859156] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_connected addr=lb:500515326worker1 # [ 136.865007] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5327lb # [ 137.031217] niks3-server[1178]: 2026/09/23 16:14:43 INFO lead: acquired remote=192.168.1.3:472985328worker1 # [ 137.033088] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_take_over5329worker1 # [ 137.067884] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_connected addr=lb:500515330worker1 # [ 137.072301] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5331worker1 # [ 137.268833] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_connected addr=lb:500515332worker1 # [ 137.274256] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5333worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5334worker1 # [ 137.475139] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_connected addr=lb:500515335worker1 # [ 137.485920] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5336worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5337worker1 # [ 137.683382] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:44Z level=info event=scheduler_connected addr=lb:500515338worker1 # [ 137.703105] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:408945339worker1 # [ 137.706015] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05340worker1 # [ 137.721581] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:44Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:409105341worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5342worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5343client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25344client # don't know how to build these paths:5345client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5346worker1 # [ 138.816737] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556685347worker1 # [ 138.823979] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556685348worker1 # [ 138.852168] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05349worker1 # [ 138.859596] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15350worker1 # [ 138.869279] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556685351worker1 # [ 138.880912] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556685352client # copying 1 paths to 'lb:50051'...5353worker1 # [ 138.896050] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556685354client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5355worker1 # [ 138.905762] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5356worker1 # [ 138.923378] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=info event=niks3_push_started pid=14065357worker1 # [ 139.062809] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.400Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5358worker1 # [ 139.174194] nix-daemon[856]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5359worker1 # [ 139.194092] nix-daemon[856]: reaped child process 1421, status = succeeded5360lb # [ 139.197900] niks3-server[1178]: 2026/09/23 16:14:45 INFO Received push request method=POST path=/api/pushes5361worker1 # [ 139.214326] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.551Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5362worker1 # [ 139.215892] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.551Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5363lb # [ 139.248124] niks3-server[1178]: 2026/09/23 16:14:45 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5364worker1 # [ 139.259286] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.596Z level=INFO msg="Uploading 1 narinfos"5365lb # [ 139.258600] niks3-server[1178]: 2026/09/23 16:14:45 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5366lb # [ 139.270168] niks3-server[1178]: 2026/09/23 16:14:45 INFO Signed narinfos id=41 count=15367lb # [ 139.277947] niks3-server[1178]: 2026/09/23 16:14:45 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5368lb # [ 139.296172] niks3-server[1178]: 2026/09/23 16:14:45 INFO Received complete push request method=POST path=/api/pushes/41/complete5369lb # [ 139.300728] envoy[895]: conn peer=[2001:db8:1::1]:52668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9749 tx=4478 ms=61055370lb # [ 139.304455] envoy[895]: conn peer=[2001:db8:1::4]:51300 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=13179 tx=1012 ms=171115371worker1 # [ 139.310896] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.649Z level=INFO msg="Upload complete. (244ms)"5372lb # [ 139.310511] niks3-server[1178]: 2026/09/23 16:14:45 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5373worker1 # [ 139.320451] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=0 paths=1 nar_bytes_in=5205374worker1 # [ 139.341048] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556685375worker1 # [ 139.348385] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5376worker1 # [ 139.354251] nix-daemon[856]: reaped child process 1404, status = succeeded5377client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5378worker1 # [ 139.367239] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5379client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5380client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5381client # farm-a> LOG-farm-a-on-standby5382worker1 # [ 140.610158] nix-daemon[856]: reaped child process 1425, status = succeeded5383worker1 # [ 140.728226] nix-daemon[856]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5384worker1 # [ 140.746413] nix-daemon[856]: reaped child process 1444, status = succeeded5385lb # [ 140.750240] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received push request method=POST path=/api/pushes5386worker1 # [ 140.764403] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.101Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5387worker1 # [ 140.765959] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.101Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5388lb # [ 140.774872] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5389lb # [ 140.785608] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5390worker1 # [ 140.795971] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.134Z level=INFO msg="Uploading 1 narinfos"5391lb # [ 140.790922] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5392lb # [ 140.796332] niks3-server[1178]: 2026/09/23 16:14:47 INFO Signed narinfos id=42 count=15393lb # [ 140.802365] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5394lb # [ 140.811368] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received complete push request method=POST path=/api/pushes/42/complete5395lb # [ 140.816427] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5396worker1 # [ 140.817979] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.156Z level=INFO msg="Upload complete. (233ms)"5397worker1 # [ 140.835436] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15398worker1 # [ 140.849381] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05399worker1 # [ 140.853371] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15400worker1 # [ 140.857507] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556685401worker1 # [ 140.861999] nix-daemon[856]: reaped child process 1423, status = succeeded5402worker1 # [ 140.865093] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556685403worker1 # [ 140.872985] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.211Z level=INFO msg="All 1 paths already cached"5404client # copying 1 paths to 'lb:50051'...5405client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5406worker1 # [ 140.879130] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556685407worker1 # [ 140.888741] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5408worker1 # [ 141.012463] nix-daemon[856]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5409worker1 # [ 141.031341] nix-daemon[856]: reaped child process 1453, status = succeeded5410lb # [ 141.032964] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received push request method=POST path=/api/pushes5411worker1 # [ 141.044762] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.383Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5412worker1 # [ 141.046227] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.383Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5413lb # [ 141.062629] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5414worker1 # [ 141.077692] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.416Z level=INFO msg="Uploading 1 narinfos"5415lb # [ 141.076756] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5416lb # [ 141.085052] niks3-server[1178]: 2026/09/23 16:14:47 INFO Signed narinfos id=43 count=15417lb # [ 141.090493] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5418worker1 # [ 141.105361] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.444Z level=INFO msg="Upload complete. (206ms)"5419lb # [ 141.104315] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received complete push request method=POST path=/api/pushes/43/complete5420worker1 # [ 141.112214] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=0 paths=1 nar_bytes_in=6405421lb # [ 141.111813] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5422worker1 # [ 141.120698] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556685423worker1 # [ 141.130879] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5424worker1 # [ 141.135513] nix-daemon[856]: reaped child process 1448, status = succeeded5425worker1 # [ 141.145923] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5426client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5427client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5428client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5429client # farm-b> LOG-farm-b-on-standby5430worker1 # [ 142.650187] nix-daemon[856]: reaped child process 1457, status = succeeded5431worker1 # [ 142.728354] nix-daemon[856]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5432worker1 # [ 142.747707] nix-daemon[856]: reaped child process 1476, status = succeeded5433lb # [ 142.753311] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received push request method=POST path=/api/pushes5434worker1 # [ 142.766323] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.103Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5435worker1 # [ 142.767955] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.103Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5436lb # [ 142.789188] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5437lb # [ 142.804132] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5438worker1 # [ 142.820136] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.158Z level=INFO msg="Uploading 1 narinfos"5439lb # [ 142.812895] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5440lb # [ 142.822722] niks3-server[1178]: 2026/09/23 16:14:49 INFO Signed narinfos id=44 count=15441lb # [ 142.832434] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received complete push request method=POST path=/api/pushes/44/complete5442worker1 # [ 142.846959] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.185Z level=INFO msg="Upload complete. (288ms)"5443lb # [ 142.847885] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5444worker1 # [ 142.869896] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15445lb # [ 142.876168] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5446worker1 # [ 142.888555] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05447worker1 # [ 142.893345] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15448worker1 # [ 142.901066] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556685449worker1 # [ 142.907567] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556685450worker1 # [ 142.913364] nix-daemon[856]: reaped child process 1455, status = succeeded5451worker1 # [ 142.916836] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.255Z level=INFO msg="All 2 paths already cached"5452client # copying 1 paths to 'lb:50051'...5453client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5454worker1 # [ 142.927108] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556685455worker1 # [ 142.939958] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5456worker1 # [ 143.080496] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)5457worker1 # [ 143.097524] nix-daemon[856]: reaped child process 1485, status = succeeded5458lb # [ 143.102939] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received push request method=POST path=/api/pushes5459worker1 # [ 143.118216] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.456Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5460worker1 # [ 143.119808] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.456Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5461lb # [ 143.139348] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5462worker1 # [ 143.149918] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.488Z level=INFO msg="Uploading 1 narinfos"5463lb # [ 143.154888] niks3-server[1178]: 2026/09/23 16:14:49 INFO Signed narinfos id=45 count=15464lb # [ 143.163403] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5465lb # [ 143.171382] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5466lb # [ 143.187597] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received complete push request method=POST path=/api/pushes/45/complete5467worker1 # [ 143.194292] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.532Z level=INFO msg="Upload complete. (242ms)"5468lb # [ 143.194849] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5469worker1 # [ 143.205398] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=0 paths=1 nar_bytes_in=7685470worker1 # [ 143.220565] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556685471worker1 # [ 143.230149] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5472worker1 # [ 143.239409] nix-daemon[856]: reaped child process 1480, status = succeeded5473client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5474worker1 # [ 143.248679] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5475client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5476client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5477client # farm-top> LOG-farm-top-on-standby5478worker1 # [ 144.717506] nix-daemon[856]: reaped child process 1489, status = succeeded5479worker1 # [ 144.831924] nix-daemon[856]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)5480worker1 # [ 144.854685] nix-daemon[856]: reaped child process 1508, status = succeeded5481lb # [ 144.857157] niks3-server[1178]: 2026/09/23 16:14:51 INFO Received push request method=POST path=/api/pushes5482worker1 # [ 144.872574] nix-grpc-daemon[1406]: time=2026-09-23T16:14:51.208Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5483worker1 # [ 144.874257] nix-grpc-daemon[1406]: time=2026-09-23T16:14:51.208Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5484lb # [ 144.903905] niks3-server[1178]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5485worker1 # [ 144.918299] nix-grpc-daemon[1406]: time=2026-09-23T16:14:51.256Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 144.919337] niks3-server[1178]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5487lb # [ 144.928240] niks3-server[1178]: 2026/09/23 16:14:51 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign5488lb # [ 144.940499] niks3-server[1178]: 2026/09/23 16:14:51 INFO Signed narinfos id=46 count=15489lb # [ 144.942868] niks3-server[1178]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5490lb # [ 144.950279] niks3-server[1178]: 2026/09/23 16:14:51 INFO Received complete push request method=POST path=/api/pushes/46/complete5491worker1 # [ 144.957923] nix-grpc-daemon[1406]: time=2026-09-23T16:14:51.296Z level=INFO msg="Upload complete. (297ms)"5492lb # [ 144.957510] niks3-server[1178]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5493worker1 # [ 144.973414] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15494worker1 # [ 144.992784] nix-daemon[856]: reaped child process 1487, status = succeeded5495client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.43 seconds)5496client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5497client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5498worker2 # [ 145.289655] systemd[1]: Starting scheduler-token.service...5499worker2 # [ 145.362833] systemd[1]: scheduler-token.service: Deactivated successfully.5500worker2 # [ 145.371687] systemd[1]: Finished scheduler-token.service.5501worker2 # [ 145.383038] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 77ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5502worker2 # [ 145.390759] systemd[1]: Starting Nix worker-protocol over gRPC...5503worker2 # [ 145.497642] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5504worker2 # [ 145.499926] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5505worker2 # [ 145.503827] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15506worker2 # [ 145.510748] nix-grpc-daemon[1365]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5507worker2 # [ 145.532070] nix-grpc-daemon[1365]: E0000 00:00:1790180091.833597 1365 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5508worker2 # [ 145.534422] nix-grpc-daemon[1365]: E0000 00:00:1790180091.835752 1365 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5509lb: must succeed: curl -sf localhost:9901/clusters5510worker2 # [ 145.559218] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5511worker2 # [ 145.561593] nix-grpc-daemon[1365]: E0000 00:00:1790180091.845489 1405 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5512worker2 # [ 145.564193] nix-grpc-daemon[1365]: E0000 00:00:1790180091.867584 1405 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5513worker2 # [ 145.566424] nix-grpc-daemon[1365]: E0000 00:00:1790180091.867752 1405 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5514worker2 # [ 145.568805] nix-grpc-daemon[1365]: E0000 00:00:1790180091.883334 1405 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5515worker2 # [ 145.571363] systemd[1]: Started Nix worker-protocol over gRPC.5516worker2 # [ 145.585423] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=scheduler_connected addr=lb:500515517worker1 # [ 145.586365] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:51Z level=info event=oidc_jwks provider=mock keys=15518worker1 # [ 145.594271] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:408945519worker1 # [ 145.603496] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05520lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5521worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5522worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5523lb: must succeed: curl -sf localhost:9901/clusters5524lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5525lb: must succeed: curl -sf localhost:9901/clusters5526lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5527worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5528worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5529(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.90 seconds)5530subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5531client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5532client # [ 145.951543] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5533client # Running as unit: rs.service; invocation ID: 06ed96ef1c71481cbac461927da43ba35534client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5535worker1 # [ 146.134060] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556685536client # [ 146.153630] nix[1401]: don't know how to build these paths:5537client # [ 146.154871] nix[1401]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5538worker1 # [ 146.155347] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05539worker1 # [ 146.165449] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15540worker1 # [ 146.174484] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556685541worker1 # [ 146.182438] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556685542client # [ 146.202951] nix[1401]: copying 1 paths to 'lb:50051'...5543client # [ 146.206970] nix[1401]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5544worker1 # [ 146.195548] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556685545worker1 # [ 146.210606] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5546worker1 # [ 146.365571] nix-daemon[856]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)5547worker1 # [ 146.389208] nix-daemon[856]: reaped child process 1524, status = succeeded5548lb # [ 146.394033] niks3-server[1178]: 2026/09/23 16:14:52 INFO Received push request method=POST path=/api/pushes5549worker1 # [ 146.405093] nix-grpc-daemon[1406]: time=2026-09-23T16:14:52.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5550worker1 # [ 146.406599] nix-grpc-daemon[1406]: time=2026-09-23T16:14:52.743Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5551lb # [ 146.420965] niks3-server[1178]: 2026/09/23 16:14:52 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign5552worker1 # [ 146.429955] nix-grpc-daemon[1406]: time=2026-09-23T16:14:52.768Z level=INFO msg="Uploading 1 narinfos"5553lb # [ 146.435425] niks3-server[1178]: 2026/09/23 16:14:52 INFO Signed narinfos id=47 count=15554lb # [ 146.445083] niks3-server[1178]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5555lb # [ 146.452842] niks3-server[1178]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5556lb # [ 146.466023] niks3-server[1178]: 2026/09/23 16:14:52 INFO Received complete push request method=POST path=/api/pushes/47/complete5557lb # [ 146.472521] niks3-server[1178]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5558worker1 # [ 146.474105] nix-grpc-daemon[1406]: time=2026-09-23T16:14:52.812Z level=INFO msg="Upload complete. (249ms)"5559worker1 # [ 146.483914] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=0 paths=1 nar_bytes_in=4165560worker1 # [ 146.500395] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556685561worker1 # [ 146.507373] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5562worker1 # [ 146.513261] nix-daemon[856]: reaped child process 1519, status = succeeded5563client # [ 146.536678] nix[1401]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5564worker1 # [ 146.528966] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5565lb: must succeed: curl -sf localhost:9901/clusters5566lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5567lb: must succeed: curl -sf localhost:9901/clusters5568lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5569lb: must succeed: curl -sf localhost:9901/clusters5570lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5571lb: must succeed: curl -sf localhost:9901/clusters5572lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5573worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5574worker2 # [ 147.630984] systemd-journald[271]: Received client request to rotate journal, rotating.5575worker2 # [ 147.645929] systemd-journald[271]: Vacuuming done, freed 0B of archived journals from /var/log/journal/200489c2afc34ee798e31e248631b567.5576worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5577worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5578worker1 # [ 147.661547] systemd[1]: Stopping Nix worker-protocol over gRPC...5579worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5580client: must succeed: sleep 5.05581worker1 # [ 147.737822] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:54Z level=info event=draining5582lb # [ 149.267600] envoy[895]: conn peer=[2001:db8:1::1]:37216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9606 tx=4443 ms=61665583client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5584worker1 # [ 152.963092] nix-daemon[856]: reaped child process 1528, status = succeeded5585worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5586worker1 # [ 153.054291] nix-daemon[856]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)5587worker1 # [ 153.073941] nix-daemon[856]: reaped child process 1568, status = succeeded5588lb # [ 153.077690] niks3-server[1178]: 2026/09/23 16:14:59 INFO Received push request method=POST path=/api/pushes5589worker1 # [ 153.091282] nix-grpc-daemon[1406]: time=2026-09-23T16:14:59.429Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5590worker1 # [ 153.095478] nix-grpc-daemon[1406]: time=2026-09-23T16:14:59.431Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5591lb # [ 153.122795] niks3-server[1178]: 2026/09/23 16:14:59 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign5592worker1 # [ 153.134381] nix-grpc-daemon[1406]: time=2026-09-23T16:14:59.472Z level=INFO msg="Uploading 1 narinfos"5593lb # [ 153.139112] niks3-server[1178]: 2026/09/23 16:14:59 INFO Signed narinfos id=48 count=15594lb # [ 153.148925] niks3-server[1178]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5595lb # [ 153.159231] niks3-server[1178]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5596lb # [ 153.172537] niks3-server[1178]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5597lb # [ 153.175729] niks3-server[1178]: 2026/09/23 16:14:59 INFO Received complete push request method=POST path=/api/pushes/48/complete5598lb # [ 153.181065] niks3-server[1178]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5599worker1 # [ 153.183086] nix-grpc-daemon[1406]: time=2026-09-23T16:14:59.521Z level=INFO msg="Upload complete. (238ms)"5600worker1 # [ 153.197171] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=6 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15601worker1 # [ 153.214785] nix-daemon[856]: reaped child process 1526, status = succeeded5602client # [ 153.240689] systemd[1]: rs.service: Deactivated successfully.5603client # [ 153.245070] systemd[1]: rs.service: Consumed 185ms CPU time over 7.293s wall clock time, 39.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.5604worker1 # [ 153.739775] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:00Z level=info event=signal_exit5605lb # [ 154.036242] niks3-server[1178]: 2026/09/23 16:15:00 INFO lead: released remote=192.168.1.3:472985606worker1 # [ 155.041122] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=scheduler_restarting5607worker2 # [ 155.058435] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_restarting addr=lb:500515608lb # [ 155.038497] envoy[895]: [2026-09-23 16:15:01.361][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5609worker1 # [ 155.042831] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=scheduler_restarting addr=lb:500515610worker1 # [ 155.051825] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:408945611worker1 # [ 155.061656] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=worker_gone addr=192.168.1.4:500515612worker1 # [ 155.066340] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:408945613worker1 # [ 155.071458] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=worker_gone addr=192.168.1.3:500515614worker1 # [ 155.097464] nix-daemon[856]: reaped child process 1379, status = succeeded5615worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.17 seconds)5616client: waiting for success: ! systemctl is-active rs5617client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5618client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5619worker2 # [ 155.258859] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_connected addr=lb:500515620worker2 # [ 155.264530] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5621client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5622client: must fail: journalctl -u rs | grep -q 'reconnecting'5623worker1 # [ 155.266728] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5624client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5625lb: must succeed: curl -sf localhost:9901/clusters5626lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5627worker2 # [ 155.465661] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_connected addr=lb:500515628worker2 # [ 155.470962] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5629lb # [ 155.507078] niks3-server[1178]: 2026/09/23 16:15:01 INFO lead: acquired remote=192.168.1.4:541605630worker2 # [ 155.528569] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_take_over5631worker2 # [ 155.672170] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_connected addr=lb:500515632worker2 # [ 155.677418] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5633worker1 # [ 155.746902] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5634worker1 # [ 155.755558] systemd[1]: Stopped Nix worker-protocol over gRPC.5635worker1 # [ 155.761904] systemd[1]: nix-grpc-daemon.service: Consumed 1.560s CPU time over 31.193s wall clock time, 39.3M memory peak, 331.7K incoming IP traffic, 146.9K outgoing IP traffic.5636worker1 # [ 155.788507] systemd[1]: Starting Nix worker-protocol over gRPC...5637worker2 # [ 155.878689] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:500515638worker2 # [ 155.884210] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5639worker1 # [ 155.905662] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5640worker1 # [ 155.907891] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5641worker1 # [ 155.911699] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15642worker1 # [ 155.917326] nix-grpc-daemon[1576]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5643worker1 # [ 155.918758] nix-grpc-daemon[1576]: E0000 00:00:1790180102.255151 1576 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5644worker1 # [ 155.922579] nix-grpc-daemon[1576]: E0000 00:00:1790180102.261060 1576 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5645worker1 # [ 155.961624] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5646worker1 # [ 155.963796] nix-grpc-daemon[1576]: E0000 00:00:1790180102.269642 1617 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5647worker1 # [ 155.966659] nix-grpc-daemon[1576]: E0000 00:00:1790180102.284882 1617 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5648worker1 # [ 155.972141] nix-grpc-daemon[1576]: E0000 00:00:1790180102.285034 1617 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5649worker1 # [ 155.976663] nix-grpc-daemon[1576]: E0000 00:00:1790180102.299528 1617 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5650worker1 # [ 155.978974] systemd[1]: Started Nix worker-protocol over gRPC.5651worker1 # [ 155.979786] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:500515652worker1 # [ 155.981209] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5653worker2 # [ 156.087315] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:500515654worker2 # [ 156.091565] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5655worker1 # [ 156.178038] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:500515656worker1 # [ 156.183316] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5657worker2 # [ 156.288242] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:500515658worker2 # [ 156.293489] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5659lb: must succeed: curl -sf localhost:9901/clusters5660worker1 # [ 156.384196] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:500515661worker1 # [ 156.389609] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5662lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5663worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5664worker2 # [ 156.494469] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:500515665worker2 # [ 156.505879] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5666worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5667worker1 # [ 156.590580] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:500515668worker2 # [ 156.611567] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:327685669worker2 # [ 156.617097] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05670worker2 # [ 156.701050] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=scheduler_connected addr=lb:500515671worker2 # [ 156.714262] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=oidc_jwks provider=mock keys=15672worker2 # [ 156.719041] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:327685673worker2 # [ 156.725099] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05674worker2 # [ 156.901842] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:327825675lb: must succeed: curl -sf localhost:9901/clusters5676lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5677worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5678worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5679(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.76 seconds)5680subtest: scheduler restart mid-build: build finishes, no double build5681client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5682client # [ 157.705438] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5683client # Running as unit: mid.service; invocation ID: 93fc0abcbbca40929cea39fc562be9235684client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5685worker2 # [ 157.905313] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:327965686worker2 # [ 157.912347] nix-daemon[844]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5687client # [ 157.919010] nix[1437]: don't know how to build these paths:5688client # [ 157.920424] nix[1437]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5689worker2 # [ 157.932794] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:04Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05690worker2 # [ 157.937552] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:04Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15691worker1 # [ 157.942654] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470965692worker1 # [ 157.957306] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5693worker1 # [ 157.980953] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)5694worker1 # [ 158.082598] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 2/5)5695worker1 # [ 158.140391] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 3/5)5696worker1 # [ 158.530678] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 672 ms (attempt 4/5)5697worker1 # [ 159.205020] nix-grpc-daemon[1576]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5698worker1 # [ 159.205248] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5699worker1 # [ 159.225629] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470965700client # [ 159.253416] nix[1437]: copying 1 paths to 'lb:50051'...5701worker1 # [ 159.241606] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470965702client # [ 159.258519] nix[1437]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5703worker1 # [ 159.257081] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5704lb # [ 159.267039] envoy[895]: conn peer=[2001:db8:1::1]:59704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1516 ms=70765705lb # [ 159.272217] envoy[895]: conn peer=[2001:db8:1::3]:39548 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2473 tx=870 ms=304805706worker1 # [ 159.291056] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:05Z level=info event=niks3_push_started pid=16445707worker1 # [ 159.425801] nix-grpc-daemon[1644]: time=2026-09-23T16:15:05.764Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5708worker1 # [ 159.540275] nix-daemon[856]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)5709worker1 # [ 159.558446] nix-daemon[856]: reaped child process 1659, status = succeeded5710lb # [ 159.565634] niks3-server[1178]: 2026/09/23 16:15:05 INFO Received push request method=POST path=/api/pushes5711worker1 # [ 159.580068] nix-grpc-daemon[1644]: time=2026-09-23T16:15:05.917Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5712worker1 # [ 159.581599] nix-grpc-daemon[1644]: time=2026-09-23T16:15:05.917Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5713worker1 # [ 159.624084] nix-grpc-daemon[1644]: time=2026-09-23T16:15:05.962Z level=INFO msg="Uploading 1 narinfos"5714lb # [ 159.611931] niks3-server[1178]: 2026/09/23 16:15:05 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign5715lb # [ 159.631747] niks3-server[1178]: 2026/09/23 16:15:05 INFO Signed narinfos id=49 count=15716lb # [ 159.644389] niks3-server[1178]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5717lb # [ 159.660572] niks3-server[1178]: 2026/09/23 16:15:05 INFO Received complete push request method=POST path=/api/pushes/49/complete5718lb # [ 159.672855] niks3-server[1178]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5719lb # [ 159.679270] niks3-server[1178]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5720worker1 # [ 159.685696] nix-grpc-daemon[1644]: time=2026-09-23T16:15:06.024Z level=INFO msg="Upload complete. (256ms)"5721worker1 # [ 159.696646] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47096 duration_s=0 paths=1 nar_bytes_in=4245722worker1 # [ 159.712922] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470965723worker1 # [ 159.724806] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5724worker1 # [ 159.731334] nix-daemon[856]: reaped child process 1642, status = succeeded5725worker1 # [ 159.743391] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5726client # [ 159.753798] nix[1437]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5727lb: must succeed: curl -sf localhost:9901/clusters5728lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5729lb: must succeed: curl -sf localhost:9901/clusters5730lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5731worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5732worker2 # [ 161.578800] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1365 (nix-grpc-daemon) on client request.5733lb # [ 161.573864] niks3-server[1178]: 2026/09/23 16:15:07 INFO lead: released remote=192.168.1.4:541605734worker2 # [ 161.593144] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5735lb # [ 161.579548] envoy[895]: [2026-09-23 16:15:07.903][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5736worker1 # [ 161.593051] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"5737client # [ 161.606426] nix[1437]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5738worker2 # [ 161.621092] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5739worker2 # [ 161.623877] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5740worker2 # [ 161.648343] systemd[1]: nix-grpc-daemon.service: Consumed 278ms CPU time over 16.228s wall clock time, 14.9M memory peak, 152.5K incoming IP traffic, 50.1K outgoing IP traffic.5741worker2 # [ 161.651457] systemd[1]: Starting scheduler-token.service...5742worker2 # [ 161.654986] nix-daemon[844]: reaped child process 1473, status = succeeded5743worker2 # [ 161.704544] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5744worker2 # [ 161.711726] systemd[1]: scheduler-token.service: Deactivated successfully.5745worker2 # [ 161.713931] systemd[1]: Finished scheduler-token.service.5746worker2 # [ 161.715091] systemd[1]: scheduler-token.service: Consumed 31ms CPU time over 69ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5747worker2 # [ 161.727006] systemd[1]: Starting Nix worker-protocol over gRPC...5748worker1 # [ 161.793120] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:08Z level=info event=scheduler_connected addr=lb:500515749worker2 # [ 161.823985] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5750worker2 # [ 161.826272] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5751worker2 # [ 161.829988] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15752worker2 # [ 161.836056] nix-grpc-daemon[1504]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5753worker2 # [ 161.837494] nix-grpc-daemon[1504]: E0000 00:00:1790180108.159545 1504 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5754worker2 # [ 161.840889] nix-grpc-daemon[1504]: E0000 00:00:1790180108.165847 1504 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5755worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.34 seconds)5756client: must succeed: sleep 5.05757worker2 # [ 161.855042] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5758lb # [ 161.840862] niks3-server[1178]: 2026/09/23 16:15:08 INFO lead: acquired remote=192.168.1.4:406125759worker2 # [ 161.880107] systemd[1]: Started Nix worker-protocol over gRPC.5760worker2 # [ 161.880969] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=scheduler_take_over5761worker2 # [ 161.882062] nix-grpc-daemon[1504]: E0000 00:00:1790180108.184141 1544 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5762worker2 # [ 161.894993] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:583145763worker2 # [ 161.897142] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=15764worker2 # [ 161.899134] nix-grpc-daemon[1504]: E0000 00:00:1790180108.200209 1544 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5765worker2 # [ 161.901470] nix-grpc-daemon[1504]: E0000 00:00:1790180108.200368 1544 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5766worker2 # [ 161.906526] nix-grpc-daemon[1504]: E0000 00:00:1790180108.219333 1544 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5767worker2 # [ 161.911571] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=scheduler_connected addr=lb:500515768worker2 # [ 161.916862] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=oidc_jwks provider=mock keys=15769worker2 # [ 161.918154] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:583145770worker2 # [ 161.920041] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05771worker2 # [ 162.344468] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:583305772client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5773client: waiting for success: ! systemctl is-active mid5774worker1 # [ 167.181338] nix-daemon[856]: reaped child process 1663, status = succeeded5775worker1 # [ 167.291898] nix-daemon[856]: accepted connection from pid 1700, user nix-grpc-daemon (trusted)5776worker1 # [ 167.310208] nix-daemon[856]: reaped child process 1703, status = succeeded5777lb # [ 167.316981] niks3-server[1178]: 2026/09/23 16:15:13 INFO Received push request method=POST path=/api/pushes5778worker1 # [ 167.336434] nix-grpc-daemon[1644]: time=2026-09-23T16:15:13.672Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5779worker1 # [ 167.338070] nix-grpc-daemon[1644]: time=2026-09-23T16:15:13.673Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5780lb # [ 167.367304] niks3-server[1178]: 2026/09/23 16:15:13 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign5781worker1 # [ 167.377327] nix-grpc-daemon[1644]: time=2026-09-23T16:15:13.715Z level=INFO msg="Uploading 1 narinfos"5782lb # [ 167.387822] niks3-server[1178]: 2026/09/23 16:15:13 INFO Signed narinfos id=50 count=15783lb # [ 167.403253] niks3-server[1178]: 2026/09/23 16:15:13 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5784lb # [ 167.421764] niks3-server[1178]: 2026/09/23 16:15:13 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5785lb # [ 167.435968] niks3-server[1178]: 2026/09/23 16:15:13 INFO Received complete push request method=POST path=/api/pushes/50/complete5786lb # [ 167.441071] niks3-server[1178]: 2026/09/23 16:15:13 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5787worker1 # [ 167.438731] nix-grpc-daemon[1644]: time=2026-09-23T16:15:13.777Z level=INFO msg="Upload complete. (276ms)"5788lb # [ 167.448522] niks3-server[1178]: 2026/09/23 16:15:13 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5789worker1 # [ 167.452516] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47096 duration_s=7 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15790worker1 # [ 167.473297] nix-daemon[856]: reaped child process 1661, status = succeeded5791client # [ 167.501477] systemd[1]: mid.service: Deactivated successfully.5792client # [ 167.505710] systemd[1]: mid.service: Consumed 187ms CPU time over 9.799s wall clock time, 41.4M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.5793client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5794client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5795client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5796(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.69 seconds)5797subtest: interrupting the client stops the build on the worker5798client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5799client # [ 168.368891] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5800client # Running as unit: intr.service; invocation ID: 461ec0038fdc4ac59b69a6b04ff84c605801client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5802client # [ 168.547441] nix[1473]: don't know how to build these paths:5803client # [ 168.548652] nix[1473]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5804worker1 # [ 168.533624] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470965805worker2 # [ 168.564712] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:14Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05806worker2 # [ 168.572786] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:14Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15807worker2 # [ 168.583744] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583445808worker2 # [ 168.597081] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5809worker2 # [ 168.616347] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)5810worker2 # [ 168.696972] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)5811worker2 # [ 168.734554] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 3/5)5812worker2 # [ 168.774369] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 208 ms (attempt 4/5)5813worker2 # [ 168.985189] nix-grpc-daemon[1504]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5814worker2 # [ 168.992159] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5815worker2 # [ 169.008544] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583445816client # [ 169.020319] nix[1473]: copying 1 paths to 'lb:50051'...5817worker2 # [ 169.023000] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583445818client # [ 169.022654] nix[1473]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5819worker2 # [ 169.039538] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5820worker2 # [ 169.069769] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=info event=niks3_push_started pid=15785821worker2 # [ 169.223124] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.546Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5822lb # [ 169.266166] envoy[895]: conn peer=[2001:db8:1::4]:44990 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7219 tx=568 ms=160445823lb # [ 169.271416] envoy[895]: conn peer=[2001:db8:1::1]:37192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3065 tx=1606 ms=95885824worker2 # [ 169.326520] nix-daemon[844]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)5825worker2 # [ 169.344634] nix-daemon[844]: reaped child process 1594, status = succeeded5826lb # [ 169.337384] niks3-server[1178]: 2026/09/23 16:15:15 INFO Received push request method=POST path=/api/pushes5827worker2 # [ 169.368071] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.692Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5828worker2 # [ 169.369512] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.692Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5829lb # [ 169.384395] niks3-server[1178]: 2026/09/23 16:15:15 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign5830worker2 # [ 169.413072] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.736Z level=INFO msg="Uploading 1 narinfos"5831lb # [ 169.399173] niks3-server[1178]: 2026/09/23 16:15:15 INFO Signed narinfos id=51 count=15832lb # [ 169.405865] niks3-server[1178]: 2026/09/23 16:15:15 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5833lb # [ 169.413399] niks3-server[1178]: 2026/09/23 16:15:15 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5834lb # [ 169.424878] niks3-server[1178]: 2026/09/23 16:15:15 INFO Received complete push request method=POST path=/api/pushes/51/complete5835worker2 # [ 169.444408] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.769Z level=INFO msg="Upload complete. (222ms)"5836lb # [ 169.429143] niks3-server[1178]: 2026/09/23 16:15:15 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5837worker2 # [ 169.452277] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=4245838worker2 # [ 169.467469] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583445839worker2 # [ 169.476392] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5840worker2 # [ 169.480745] nix-daemon[844]: reaped child process 1576, status = succeeded5841client # [ 169.488252] nix[1473]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5842worker2 # [ 169.494793] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5843client: must succeed: systemctl kill -s INT intr5844worker2 # [ 172.012554] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:18Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5845client # [ 172.008690] nix[1473]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5846client # [ 172.017647] systemd[1]: intr.service: Sent signal SIGINT to main process 1473 (nix) on client request.5847client # [ 172.028362] nix[1473]: error: interrupted by the user5848client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5849client # [ 172.057024] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5850client # [ 172.062718] systemd[1]: intr.service: Failed with result 'exit-code'.5851client # [ 172.065548] systemd[1]: intr.service: Consumed 166ms CPU time over 3.691s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5852worker2 # [ 172.139431] nix-daemon[844]: reaped child process 1596, status = succeeded5853worker2 # [ 172.144437] nix-daemon[1598]: unexpected Nix daemon error: error: interrupted by the user5854worker2 # [ 172.156229] nix-daemon[844]: reaped child process 1598, status = failed with exit code 15855(finished: subtest: interrupting the client stops the build on the worker, in 3.83 seconds)5856subtest: a deploy mid-build drains: the build finishes, the new generation takes over5857client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5858client # [ 172.223970] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5859client # Running as unit: sw.service; invocation ID: 422122bbe6a1435ab3a89e0e5ccdcb055860client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)5861worker2 # [ 172.424076] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:583445862client # [ 172.428838] nix[1494]: don't know how to build these paths:5863client # [ 172.430467] nix[1494]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5864worker2 # [ 172.442323] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:18Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05865worker1 # [ 172.433175] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470965866worker2 # [ 172.448950] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:18Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15867worker1 # [ 172.451000] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470965868client # [ 172.477947] nix[1494]: copying 1 paths to 'lb:50051'...5869client # [ 172.479815] nix[1494]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5870worker1 # [ 172.466882] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470965871worker1 # [ 172.483371] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5872worker1 # [ 172.648531] nix-daemon[856]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)5873worker1 # [ 172.671097] nix-daemon[856]: reaped child process 1734, status = succeeded5874lb # [ 172.667945] niks3-server[1178]: 2026/09/23 16:15:18 INFO Received push request method=POST path=/api/pushes5875worker1 # [ 172.681245] nix-grpc-daemon[1644]: time=2026-09-23T16:15:19.019Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5876worker1 # [ 172.682708] nix-grpc-daemon[1644]: time=2026-09-23T16:15:19.019Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5877lb # [ 172.708663] niks3-server[1178]: 2026/09/23 16:15:19 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5878lb # [ 172.725377] niks3-server[1178]: 2026/09/23 16:15:19 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign5879worker1 # [ 172.732673] nix-grpc-daemon[1644]: time=2026-09-23T16:15:19.070Z level=INFO msg="Uploading 1 narinfos"5880lb # [ 172.732681] niks3-server[1178]: 2026/09/23 16:15:19 INFO Signed narinfos id=52 count=15881lb # [ 172.738872] niks3-server[1178]: 2026/09/23 16:15:19 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5882lb # [ 172.749273] niks3-server[1178]: 2026/09/23 16:15:19 INFO Received complete push request method=POST path=/api/pushes/52/complete5883lb # [ 172.762043] niks3-server[1178]: 2026/09/23 16:15:19 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5884worker1 # [ 172.764302] nix-grpc-daemon[1644]: time=2026-09-23T16:15:19.102Z level=INFO msg="Upload complete. (262ms)"5885worker1 # [ 172.775454] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47096 duration_s=0 paths=1 nar_bytes_in=4165886worker1 # [ 172.788579] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470965887worker1 # [ 172.801250] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5888worker1 # [ 172.807378] nix-daemon[856]: reaped child process 1729, status = succeeded5889client # [ 172.828657] nix[1494]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5890worker1 # [ 172.819740] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5891worker1: must succeed: systemctl reload nix-grpc-daemon.service5892worker1 # [ 174.865987] systemd[1]: Reloading Nix worker-protocol over gRPC...5893worker1 # [ 174.903023] systemd[1]: Reloaded Nix worker-protocol over gRPC.5894worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)5895worker1: must succeed: pgrep -f 'read -t [9]0 x'5896worker1 # [ 174.933826] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:21Z level=info event=draining5897worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)5898client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&25899worker2 # [ 175.198059] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:583445900client # don't know how to build these paths:5901client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5902worker2 # [ 175.208275] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583445903worker2 # [ 175.234993] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05904worker2 # [ 175.244317] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15905worker2 # [ 175.252747] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583445906worker2 # [ 175.260262] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583445907client # copying 1 paths to 'lb:50051'...5908worker2 # [ 175.272736] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583445909client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5910worker2 # [ 175.295150] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5911worker2 # [ 175.446563] nix-daemon[844]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)5912worker2 # [ 175.466271] nix-daemon[844]: reaped child process 1656, status = succeeded5913lb # [ 175.455351] niks3-server[1178]: 2026/09/23 16:15:21 INFO Received push request method=POST path=/api/pushes5914worker2 # [ 175.484956] nix-grpc-daemon[1578]: time=2026-09-23T16:15:21.808Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5915worker2 # [ 175.486987] nix-grpc-daemon[1578]: time=2026-09-23T16:15:21.808Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5916lb # [ 175.489226] niks3-server[1178]: 2026/09/23 16:15:21 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign5917worker2 # [ 175.514943] nix-grpc-daemon[1578]: time=2026-09-23T16:15:21.839Z level=INFO msg="Uploading 1 narinfos"5918lb # [ 175.503250] niks3-server[1178]: 2026/09/23 16:15:21 INFO Signed narinfos id=53 count=15919lb # [ 175.510452] niks3-server[1178]: 2026/09/23 16:15:21 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5920lb # [ 175.523345] niks3-server[1178]: 2026/09/23 16:15:21 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5921worker2 # [ 175.546411] nix-grpc-daemon[1578]: time=2026-09-23T16:15:21.871Z level=INFO msg="Upload complete. (234ms)"5922lb # [ 175.530345] niks3-server[1178]: 2026/09/23 16:15:21 INFO Received complete push request method=POST path=/api/pushes/53/complete5923lb # [ 175.535052] niks3-server[1178]: 2026/09/23 16:15:21 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5924worker2 # [ 175.556291] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=5285925worker2 # [ 175.568658] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583445926worker2 # [ 175.583340] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5927worker2 # [ 175.586525] nix-daemon[844]: reaped child process 1651, status = succeeded5928client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5929worker2 # [ 175.597567] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5930client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5931client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5932client # farm-a> LOG-farm-a-during-drain5933worker2 # [ 176.862186] nix-daemon[844]: reaped child process 1660, status = succeeded5934worker2 # [ 176.984606] nix-daemon[844]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)5935worker2 # [ 177.002126] nix-daemon[844]: reaped child process 1679, status = succeeded5936lb # [ 176.991323] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received push request method=POST path=/api/pushes5937worker2 # [ 177.020407] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.343Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5938worker2 # [ 177.022031] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.343Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5939lb # [ 177.041290] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5940worker2 # [ 177.068121] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.392Z level=INFO msg="Uploading 1 narinfos"5941lb # [ 177.056835] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign5942lb # [ 177.065912] niks3-server[1178]: 2026/09/23 16:15:23 INFO Signed narinfos id=54 count=15943lb # [ 177.071835] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5944lb # [ 177.083691] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5945lb # [ 177.088384] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received complete push request method=POST path=/api/pushes/54/complete5946worker2 # [ 177.104373] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.429Z level=INFO msg="Upload complete. (294ms)"5947lb # [ 177.092090] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5948worker2 # [ 177.122855] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=15949worker2 # [ 177.141458] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05950worker2 # [ 177.149307] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15951worker2 # [ 177.156634] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583445952worker2 # [ 177.163163] nix-daemon[844]: reaped child process 1658, status = succeeded5953worker2 # [ 177.167226] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583445954worker2 # [ 177.173154] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.497Z level=INFO msg="All 1 paths already cached"5955client # copying 1 paths to 'lb:50051'...5956worker2 # [ 177.184604] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583445957client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5958worker2 # [ 177.206042] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5959worker2 # [ 177.359670] nix-daemon[844]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)5960worker2 # [ 177.375997] nix-daemon[844]: reaped child process 1688, status = succeeded5961lb # [ 177.365750] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received push request method=POST path=/api/pushes5962worker2 # [ 177.390978] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.715Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5963worker2 # [ 177.392635] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.715Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5964lb # [ 177.394689] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5965worker2 # [ 177.429825] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.754Z level=INFO msg="Uploading 1 narinfos"5966lb # [ 177.404667] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign5967lb # [ 177.419051] niks3-server[1178]: 2026/09/23 16:15:23 INFO Signed narinfos id=55 count=15968lb # [ 177.425993] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5969lb # [ 177.439865] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received complete push request method=POST path=/api/pushes/55/complete5970worker2 # [ 177.463812] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.788Z level=INFO msg="Upload complete. (242ms)"5971lb # [ 177.449485] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5972worker2 # [ 177.474110] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=6405973worker2 # [ 177.490755] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583445974worker2 # [ 177.501223] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5975worker2 # [ 177.504467] nix-daemon[844]: reaped child process 1683, status = succeeded5976client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5977client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5978worker2 # [ 177.519969] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5979client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5980client # farm-b> LOG-farm-b-during-drain5981worker2 # [ 178.196353] nix-daemon[844]: reaped child process 1692, status = succeeded5982worker2 # [ 178.296921] nix-daemon[844]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)5983worker2 # [ 178.317027] nix-daemon[844]: reaped child process 1711, status = succeeded5984lb # [ 178.300707] niks3-server[1178]: 2026/09/23 16:15:24 INFO Received push request method=POST path=/api/pushes5985worker2 # [ 178.328626] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.652Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5986worker2 # [ 178.330755] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.652Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5987lb # [ 178.349536] niks3-server[1178]: 2026/09/23 16:15:24 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign5988worker2 # [ 178.375863] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.700Z level=INFO msg="Uploading 1 narinfos"5989lb # [ 178.363765] niks3-server[1178]: 2026/09/23 16:15:24 INFO Signed narinfos id=56 count=15990lb # [ 178.371085] niks3-server[1178]: 2026/09/23 16:15:24 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5991lb # [ 178.378043] niks3-server[1178]: 2026/09/23 16:15:24 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5992worker2 # [ 178.403765] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.728Z level=INFO msg="Upload complete. (274ms)"5993lb # [ 178.387823] niks3-server[1178]: 2026/09/23 16:15:24 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5994lb # [ 178.396263] niks3-server[1178]: 2026/09/23 16:15:24 INFO Received complete push request method=POST path=/api/pushes/56/complete5995lb # [ 178.400320] niks3-server[1178]: 2026/09/23 16:15:24 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5996worker2 # [ 178.420000] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=15997worker2 # [ 178.441231] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05998worker2 # [ 178.447025] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15999worker2 # [ 178.453191] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446000worker2 # [ 178.458520] nix-daemon[844]: reaped child process 1690, status = succeeded6001worker2 # [ 178.462170] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583446002worker2 # [ 178.475123] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.800Z level=INFO msg="All 2 paths already cached"6003client # copying 1 paths to 'lb:50051'...6004client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6005worker2 # [ 178.485563] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583446006worker2 # [ 178.497061] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6007worker2 # [ 178.636004] nix-daemon[844]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6008worker2 # [ 178.657607] nix-daemon[844]: reaped child process 1720, status = succeeded6009lb # [ 178.644487] niks3-server[1178]: 2026/09/23 16:15:24 INFO Received push request method=POST path=/api/pushes6010worker2 # [ 178.669999] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.994Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6011worker2 # [ 178.671493] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.994Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6012lb # [ 178.679557] niks3-server[1178]: 2026/09/23 16:15:25 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6013lb # [ 178.692511] niks3-server[1178]: 2026/09/23 16:15:25 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6014worker2 # [ 178.713822] nix-grpc-daemon[1578]: time=2026-09-23T16:15:25.038Z level=INFO msg="Uploading 1 narinfos"6015lb # [ 178.701330] niks3-server[1178]: 2026/09/23 16:15:25 INFO Signed narinfos id=57 count=16016lb # [ 178.703829] niks3-server[1178]: 2026/09/23 16:15:25 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6017lb # [ 178.719487] niks3-server[1178]: 2026/09/23 16:15:25 INFO Received complete push request method=POST path=/api/pushes/57/complete6018lb # [ 178.724376] niks3-server[1178]: 2026/09/23 16:15:25 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6019worker2 # [ 178.739954] nix-grpc-daemon[1578]: time=2026-09-23T16:15:25.065Z level=INFO msg="Upload complete. (229ms)"6020worker2 # [ 178.752721] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=7686021worker2 # [ 178.764293] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446022worker2 # [ 178.773968] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6023worker2 # [ 178.776924] nix-daemon[844]: reaped child process 1715, status = succeeded6024client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6025worker2 # [ 178.790971] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6026client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6027client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6028lb # [ 179.267463] envoy[895]: conn peer=[2001:db8:1::1]:45326 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=828 ms=34936029client # farm-top> LOG-farm-top-during-drain6030worker2 # [ 179.658587] nix-daemon[844]: reaped child process 1724, status = succeeded6031worker2 # [ 179.788190] nix-daemon[844]: accepted connection from pid 1740, user nix-grpc-daemon (trusted)6032worker2 # [ 179.809402] nix-daemon[844]: reaped child process 1743, status = succeeded6033lb # [ 179.791862] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received push request method=POST path=/api/pushes6034worker2 # [ 179.820580] nix-grpc-daemon[1578]: time=2026-09-23T16:15:26.144Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6035worker2 # [ 179.824134] nix-grpc-daemon[1578]: time=2026-09-23T16:15:26.144Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6036lb # [ 179.828233] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6037worker2 # [ 179.849216] nix-grpc-daemon[1578]: time=2026-09-23T16:15:26.174Z level=INFO msg="Uploading 1 narinfos"6038lb # [ 179.836087] niks3-server[1178]: 2026/09/23 16:15:26 INFO Signed narinfos id=58 count=16039lb # [ 179.840983] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6040lb # [ 179.847426] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6041lb # [ 179.852828] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6042lb # [ 179.860547] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received complete push request method=POST path=/api/pushes/58/complete6043lb # [ 179.869775] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6044worker2 # [ 179.885784] nix-grpc-daemon[1578]: time=2026-09-23T16:15:26.210Z level=INFO msg="Upload complete. (282ms)"6045worker2 # [ 179.901117] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=16046worker2 # [ 179.916723] nix-daemon[844]: reaped child process 1722, status = succeeded6047client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 4.97 seconds)6048client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6049client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6050worker1: must succeed: pkill -f 'read -t [9]0 x'6051worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6052client: waiting for success: ! systemctl is-active sw6053worker1 # [ 180.296457] nix-daemon[856]: reaped child process 1738, status = succeeded6054worker1 # [ 180.358243] nix-daemon[856]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6055worker1 # [ 180.375650] nix-daemon[856]: reaped child process 1782, status = succeeded6056lb # [ 180.379536] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received push request method=POST path=/api/pushes6057worker1 # [ 180.393419] nix-grpc-daemon[1644]: time=2026-09-23T16:15:26.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6058worker1 # [ 180.394904] nix-grpc-daemon[1644]: time=2026-09-23T16:15:26.731Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6059lb # [ 180.421637] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6060worker1 # [ 180.429512] nix-grpc-daemon[1644]: time=2026-09-23T16:15:26.767Z level=INFO msg="Uploading 1 narinfos"6061lb # [ 180.433678] niks3-server[1178]: 2026/09/23 16:15:26 INFO Signed narinfos id=59 count=16062lb # [ 180.435818] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6063lb # [ 180.444431] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6064lb # [ 180.456322] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received complete push request method=POST path=/api/pushes/59/complete6065worker1 # [ 180.458484] nix-grpc-daemon[1644]: time=2026-09-23T16:15:26.797Z level=INFO msg="Upload complete. (223ms)"6066lb # [ 180.459185] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6067lb # [ 180.464416] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6068worker1 # [ 180.468875] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47096 duration_s=7 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=16069worker1 # [ 180.485467] nix-daemon[856]: reaped child process 1736, status = succeeded6070client # [ 180.518246] systemd[1]: sw.service: Deactivated successfully.6071client # [ 180.524331] systemd[1]: sw.service: Consumed 192ms CPU time over 8.294s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6072worker1 # [ 180.936150] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:27Z level=info event=signal_exit6073client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6074client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6075client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6076worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6077worker1 # [ 182.943686] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:29Z level=info event=scheduler_restarting6078worker2 # [ 182.957320] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:583146079worker2 # [ 182.965788] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:29Z level=info event=worker_gone addr=192.168.1.3:500516080worker1 # [ 182.968439] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6081worker1 # [ 182.996916] nix-daemon[856]: reaped child process 1632, status = succeeded6082worker1 # [ 183.955584] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6083worker1 # [ 183.962863] systemd[1]: nix-grpc-daemon.service: Consumed 933ms CPU time over 28.180s wall clock time, 36.5M memory peak, 56.5K incoming IP traffic, 67.2K outgoing IP traffic.6084worker1 # [ 183.978049] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6085worker1 # [ 183.999904] systemd[1]: Starting Nix worker-protocol over gRPC...6086worker1 # [ 184.122262] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6087worker1 # [ 184.124637] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6088worker1 # [ 184.128413] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16089worker1 # [ 184.134812] nix-grpc-daemon[1797]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6090worker1 # [ 184.156155] nix-grpc-daemon[1797]: E0000 00:00:1790180130.471353 1797 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6091worker1 # [ 184.158534] nix-grpc-daemon[1797]: E0000 00:00:1790180130.473343 1797 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6092worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.86 seconds)6093lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6094lb: must succeed: curl -sf localhost:9901/clusters6095worker1 # [ 184.183925] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6096worker1 # [ 184.185919] nix-grpc-daemon[1797]: E0000 00:00:1790180130.482344 1837 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6097worker1 # [ 184.188231] nix-grpc-daemon[1797]: E0000 00:00:1790180130.503925 1837 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6098worker1 # [ 184.193565] nix-grpc-daemon[1797]: E0000 00:00:1790180130.504127 1837 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6099worker1 # [ 184.197885] nix-grpc-daemon[1797]: E0000 00:00:1790180130.520330 1837 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6100worker1 # [ 184.200361] systemd[1]: Started Nix worker-protocol over gRPC.6101worker1 # [ 184.201203] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=scheduler_connected addr=lb:500516102worker2 # [ 184.216905] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:583146103worker2 # [ 184.217209] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06104lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6105lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.08 seconds)6106lb: must succeed: curl -sf localhost:9901/clusters6107lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6108worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6109worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6110(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.23 seconds)6111subtest: queue gauges show a burst while every build blocks6112lb: must succeed: curl -sf localhost:9901/clusters6113lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6114lb: must succeed: curl -sf localhost:9901/clusters6115lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6116worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6117worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6118client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6119client # [ 184.626549] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6120client # Running as unit: burst.service; invocation ID: 9c4fafbb486e481ba50306fbe0438ea76121client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6122lb: must succeed: curl -sf localhost:9901/clusters6123lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6124lb: must succeed: curl -sf localhost:9901/clusters6125lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6126worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6127worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6128worker2 # [ 184.842025] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:583446129client # [ 184.846490] nix[1550]: don't know how to build these paths:6130client # [ 184.847495] nix[1550]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6131client # [ 184.848676] nix[1550]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6132client # [ 184.849835] nix[1550]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6133client # [ 184.850898] nix[1550]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6134worker2 # [ 184.865095] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06135worker2 # [ 184.872511] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.4:50051 clients=16136worker1 # [ 184.866026] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626137worker2 # [ 184.881807] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06138worker2 # [ 184.889706] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.3:50051 clients=16139worker1 # [ 184.881478] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6140worker2 # [ 184.898547] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06141client # [ 184.904397] nix[1550]: copying 1 paths to 'lb:50051'...6142worker2 # [ 184.904915] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06143client # [ 184.907505] nix[1550]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6144worker2 # [ 184.913380] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446145worker1 # [ 184.905714] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)6146worker2 # [ 184.924767] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583446147worker2 # [ 184.934405] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583446148worker2 # [ 184.941310] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6149worker1 # [ 184.927593] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 2/5)6150worker1 # [ 184.972510] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 230 ms (attempt 3/5)6151worker2 # [ 185.067128] nix-daemon[844]: accepted connection from pid 1758, user nix-grpc-daemon (trusted)6152worker2 # [ 185.083854] nix-daemon[844]: reaped child process 1761, status = succeeded6153lb # [ 185.075221] niks3-server[1178]: 2026/09/23 16:15:31 INFO Received push request method=POST path=/api/pushes6154worker2 # [ 185.117314] nix-grpc-daemon[1578]: time=2026-09-23T16:15:31.441Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6155worker2 # [ 185.119138] nix-grpc-daemon[1578]: time=2026-09-23T16:15:31.441Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6156lb # [ 185.120654] niks3-server[1178]: 2026/09/23 16:15:31 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6157worker2 # [ 185.143820] nix-grpc-daemon[1578]: time=2026-09-23T16:15:31.468Z level=INFO msg="Uploading 1 narinfos"6158lb # [ 185.134462] niks3-server[1178]: 2026/09/23 16:15:31 INFO Signed narinfos id=60 count=16159lb # [ 185.142023] niks3-server[1178]: 2026/09/23 16:15:31 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6160lb # [ 185.154005] niks3-server[1178]: 2026/09/23 16:15:31 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6161worker2 # [ 185.179340] nix-grpc-daemon[1578]: time=2026-09-23T16:15:31.504Z level=INFO msg="Upload complete. (235ms)"6162lb # [ 185.164501] niks3-server[1178]: 2026/09/23 16:15:31 INFO Received complete push request method=POST path=/api/pushes/60/complete6163lb # [ 185.167567] niks3-server[1178]: 2026/09/23 16:15:31 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6164worker2 # [ 185.188401] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=4166165worker2 # [ 185.208831] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446166worker2 # [ 185.215785] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6167worker2 # [ 185.221517] nix-daemon[844]: reaped child process 1756, status = succeeded6168worker1 # [ 185.205644] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 334 ms (attempt 4/5)6169client # [ 185.229000] nix[1550]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6170worker2 # [ 185.235112] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6171worker1 # [ 185.542009] nix-grpc-daemon[1797]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6172worker1 # [ 185.548005] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6173worker1 # [ 185.565730] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626174client # [ 185.592289] nix[1550]: copying 1 paths to 'lb:50051'...6175worker1 # [ 185.581602] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626176client # [ 185.597002] nix[1550]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6177worker1 # [ 185.596104] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6178worker1 # [ 185.624513] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:31Z level=info event=niks3_push_started pid=18576179worker1 # [ 185.772687] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.110Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6180lb: must succeed: curl -sf localhost:9901/clusters6181worker1 # [ 185.880041] nix-daemon[856]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6182lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6183lb: must succeed: curl -sf localhost:9901/clusters6184worker1 # [ 185.899285] nix-daemon[856]: reaped child process 1873, status = succeeded6185lb # [ 185.902100] niks3-server[1178]: 2026/09/23 16:15:32 INFO Received push request method=POST path=/api/pushes6186worker1 # [ 185.920073] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.258Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6187worker1 # [ 185.921591] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.258Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6188lb # [ 185.957167] niks3-server[1178]: 2026/09/23 16:15:32 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6189worker1 # [ 185.965619] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.303Z level=INFO msg="Uploading 1 narinfos"6190lb # [ 185.968069] niks3-server[1178]: 2026/09/23 16:15:32 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6191lb # [ 185.970159] niks3-server[1178]: 2026/09/23 16:15:32 INFO Signed narinfos id=61 count=16192lb # [ 185.971604] niks3-server[1178]: 2026/09/23 16:15:32 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6193lb # [ 185.985569] niks3-server[1178]: 2026/09/23 16:15:32 INFO Received complete push request method=POST path=/api/pushes/61/complete6194lb # [ 185.989842] niks3-server[1178]: 2026/09/23 16:15:32 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6195worker1 # [ 185.990557] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.329Z level=INFO msg="Upload complete. (214ms)"6196lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)6197worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6198worker1 # [ 185.999762] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=4166199worker1 # [ 186.014628] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626200worker1 # [ 186.020835] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6201worker1 # [ 186.026010] nix-daemon[856]: reaped child process 1855, status = succeeded6202worker1 # [ 186.037183] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6203client # [ 186.048750] nix[1550]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6204worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6205worker2 # [ 187.469924] nix-daemon[844]: reaped child process 1765, status = succeeded6206worker1 # [ 187.504890] nix-daemon[856]: reaped child process 1877, status = succeeded6207worker2 # [ 187.562773] nix-daemon[844]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6208worker2 # [ 187.585781] nix-daemon[844]: reaped child process 1796, status = succeeded6209lb # [ 187.568886] niks3-server[1178]: 2026/09/23 16:15:33 INFO Received push request method=POST path=/api/pushes6210worker2 # [ 187.594163] nix-grpc-daemon[1578]: time=2026-09-23T16:15:33.918Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6211worker2 # [ 187.598341] nix-grpc-daemon[1578]: time=2026-09-23T16:15:33.920Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6212lb # [ 187.608745] niks3-server[1178]: 2026/09/23 16:15:33 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6213worker1 # [ 187.620196] nix-daemon[856]: accepted connection from pid 1902, user nix-grpc-daemon (trusted)6214worker2 # [ 187.641907] nix-grpc-daemon[1578]: time=2026-09-23T16:15:33.966Z level=INFO msg="Uploading 1 narinfos"6215lb # [ 187.625191] niks3-server[1178]: 2026/09/23 16:15:33 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6216lb # [ 187.635140] niks3-server[1178]: 2026/09/23 16:15:33 INFO Signed narinfos id=62 count=16217worker1 # [ 187.643511] nix-daemon[856]: reaped child process 1905, status = succeeded6218lb # [ 187.642269] niks3-server[1178]: 2026/09/23 16:15:33 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6219worker1 # [ 187.658662] nix-grpc-daemon[1857]: time=2026-09-23T16:15:33.995Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6220lb # [ 187.655615] niks3-server[1178]: 2026/09/23 16:15:33 INFO Received push request method=POST path=/api/pushes6221worker1 # [ 187.662111] nix-grpc-daemon[1857]: time=2026-09-23T16:15:33.995Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6222lb # [ 187.664301] niks3-server[1178]: 2026/09/23 16:15:33 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6223lb # [ 187.677082] niks3-server[1178]: 2026/09/23 16:15:33 INFO Received complete push request method=POST path=/api/pushes/62/complete6224worker2 # [ 187.701968] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.027Z level=INFO msg="Upload complete. (255ms)"6225lb # [ 187.693834] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6226lb # [ 187.703985] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6227worker2 # [ 187.718765] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=6 outputs=16228lb # [ 187.710012] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6229worker1 # [ 187.722152] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.059Z level=INFO msg="Uploading 1 narinfos"6230lb # [ 187.722570] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6231worker2 # [ 187.740372] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16232lb # [ 187.728161] niks3-server[1178]: 2026/09/23 16:15:34 INFO Signed narinfos id=63 count=16233worker2 # [ 187.749076] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446234lb # [ 187.735702] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6235worker2 # [ 187.754087] nix-daemon[844]: reaped child process 1763, status = succeeded6236worker1 # [ 187.744696] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.083Z level=INFO msg="Upload complete. (294ms)"6237lb # [ 187.744529] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received complete push request method=POST path=/api/pushes/63/complete6238worker2 # [ 187.759720] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583446239lb # [ 187.748147] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6240client # [ 187.769209] nix[1550]: copying 1 paths to 'lb:50051'...6241worker2 # [ 187.769027] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16242client # [ 187.772905] nix[1550]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6243worker1 # [ 187.757657] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=7 outputs=16244worker2 # [ 187.776224] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583446245worker2 # [ 187.788449] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6246worker1 # [ 187.779341] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626247worker1 # [ 187.785798] nix-daemon[856]: reaped child process 1875, status = succeeded6248worker1 # [ 187.793541] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626249client # [ 187.816175] nix[1550]: copying 1 paths to 'lb:50051'...6250worker1 # [ 187.805104] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626251client # [ 187.818497] nix[1550]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6252worker1 # [ 187.818507] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6253worker2 # [ 187.938193] nix-daemon[844]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6254worker2 # [ 187.962864] nix-daemon[844]: reaped child process 1803, status = succeeded6255lb # [ 187.944209] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received push request method=POST path=/api/pushes6256worker2 # [ 187.973023] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.297Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6257worker1 # [ 187.959881] nix-daemon[856]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)6258worker2 # [ 187.974898] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.297Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6259worker1 # [ 187.982325] nix-daemon[856]: reaped child process 1912, status = succeeded6260lb # [ 187.977991] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6261worker2 # [ 188.001899] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.326Z level=INFO msg="Uploading 1 narinfos"6262lb # [ 187.997469] niks3-server[1178]: 2026/09/23 16:15:34 INFO Signed narinfos id=64 count=16263lb # [ 188.005989] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received push request method=POST path=/api/pushes6264lb # [ 188.010825] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6265worker1 # [ 188.024456] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.362Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6266worker1 # [ 188.026443] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.362Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6267lb # [ 188.022663] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received complete push request method=POST path=/api/pushes/64/complete6268lb # [ 188.034821] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6269worker2 # [ 188.062640] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.387Z level=INFO msg="Upload complete. (260ms)"6270lb # [ 188.054480] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6271worker2 # [ 188.072361] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=4166272lb # [ 188.064910] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6273worker1 # [ 188.067848] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.405Z level=INFO msg="Uploading 1 narinfos"6274lb # [ 188.069378] niks3-server[1178]: 2026/09/23 16:15:34 INFO Signed narinfos id=65 count=16275lb # [ 188.076889] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6276worker2 # [ 188.092899] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446277lb # [ 188.084260] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6278worker2 # [ 188.102286] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6279worker2 # [ 188.108904] nix-daemon[844]: reaped child process 1798, status = succeeded6280lb # [ 188.094786] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received complete push request method=POST path=/api/pushes/65/complete6281worker1 # [ 188.104131] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.442Z level=INFO msg="Upload complete. (273ms)"6282lb # [ 188.103153] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6283client # [ 188.121392] nix[1550]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6284worker2 # [ 188.122357] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6285worker1 # [ 188.114752] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=4166286worker1 # [ 188.133244] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626287worker1 # [ 188.144485] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6288worker1 # [ 188.153818] nix-daemon[856]: reaped child process 1907, status = succeeded6289worker1 # [ 188.167506] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6290client # [ 188.178295] nix[1550]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6291lb # [ 189.268296] envoy[895]: conn peer=[2001:db8:1::1]:36302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4385 ms=47196292lb # [ 189.268677] envoy[895]: conn peer=[2001:db8:1::1]:36296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=1517 ms=80726293lb # [ 189.268787] envoy[895]: conn peer=[2001:db8:1::3]:34416 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1478 tx=537 ms=269856294worker1 # [ 189.836863] nix-daemon[856]: reaped child process 1916, status = succeeded6295worker1 # [ 189.954867] nix-daemon[856]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)6296worker1 # [ 189.976984] nix-daemon[856]: reaped child process 1944, status = succeeded6297lb # [ 189.981626] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received push request method=POST path=/api/pushes6298worker1 # [ 189.995659] nix-grpc-daemon[1857]: time=2026-09-23T16:15:36.334Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6299worker1 # [ 189.998136] nix-grpc-daemon[1857]: time=2026-09-23T16:15:36.336Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6300worker2 # [ 190.016975] nix-daemon[844]: reaped child process 1807, status = succeeded6301lb # [ 190.031586] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6302worker1 # [ 190.044133] nix-grpc-daemon[1857]: time=2026-09-23T16:15:36.382Z level=INFO msg="Uploading 1 narinfos"6303lb # [ 190.043374] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6304lb # [ 190.051351] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6305lb # [ 190.057509] niks3-server[1178]: 2026/09/23 16:15:36 INFO Signed narinfos id=66 count=16306lb # [ 190.061645] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6307lb # [ 190.070300] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received complete push request method=POST path=/api/pushes/66/complete6308worker1 # [ 190.075829] nix-grpc-daemon[1857]: time=2026-09-23T16:15:36.414Z level=INFO msg="Upload complete. (292ms)"6309lb # [ 190.075142] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6310worker1 # [ 190.088770] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=9 outputs=16311worker2 # [ 190.118701] nix-daemon[844]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)6312worker1 # [ 190.107271] nix-daemon[856]: reaped child process 1914, status = succeeded6313worker2 # [ 190.139277] nix-daemon[844]: reaped child process 1835, status = succeeded6314lb # [ 190.127403] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received push request method=POST path=/api/pushes6315worker2 # [ 190.152398] nix-grpc-daemon[1578]: time=2026-09-23T16:15:36.475Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6316worker2 # [ 190.153861] nix-grpc-daemon[1578]: time=2026-09-23T16:15:36.475Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6317lb # [ 190.168969] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6318worker2 # [ 190.190548] nix-grpc-daemon[1578]: time=2026-09-23T16:15:36.515Z level=INFO msg="Uploading 1 narinfos"6319lb # [ 190.179297] niks3-server[1178]: 2026/09/23 16:15:36 INFO Signed narinfos id=67 count=16320lb # [ 190.184224] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6321lb # [ 190.194125] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6322lb # [ 190.201591] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6323lb # [ 190.211342] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received complete push request method=POST path=/api/pushes/67/complete6324worker2 # [ 190.228907] nix-grpc-daemon[1578]: time=2026-09-23T16:15:36.553Z level=INFO msg="Upload complete. (286ms)"6325lb # [ 190.218013] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6326worker2 # [ 190.241569] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=8 outputs=16327worker2 # [ 190.261124] nix-daemon[844]: reaped child process 1805, status = succeeded6328client # [ 190.275195] systemd[1]: burst.service: Deactivated successfully.6329client # [ 190.279833] systemd[1]: burst.service: Consumed 233ms CPU time over 5.650s wall clock time, 49.6M memory peak, 14K incoming IP traffic, 18.6K outgoing IP traffic.6330client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6331client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6332lb: must succeed: curl -sf localhost:9901/clusters6333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6334lb: must succeed: curl -sf localhost:9901/clusters6335lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6336worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6337worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6338(finished: subtest: queue gauges show a burst while every build blocks, in 6.88 seconds)6339subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6340client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16341worker1 # [ 191.479669] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:397626342worker1 # [ 191.490307] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626343worker2 # [ 191.546713] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:37Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06344worker2 # [ 191.552908] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:37Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16345worker1 # [ 191.541992] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626346worker2 # [ 191.558988] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:37Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06347worker2 # [ 191.564388] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:37Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06348worker1 # [ 191.557419] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626349worker1 # [ 191.571794] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626350worker1 # [ 191.587298] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6351worker1 # [ 191.747525] nix-daemon[856]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)6352worker1 # [ 191.769133] nix-daemon[856]: reaped child process 1955, status = succeeded6353lb # [ 191.772696] niks3-server[1178]: 2026/09/23 16:15:38 INFO Received push request method=POST path=/api/pushes6354worker1 # [ 191.786851] nix-grpc-daemon[1857]: time=2026-09-23T16:15:38.124Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6355worker1 # [ 191.788432] nix-grpc-daemon[1857]: time=2026-09-23T16:15:38.124Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6356lb # [ 191.803108] niks3-server[1178]: 2026/09/23 16:15:38 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6357worker1 # [ 191.813534] nix-grpc-daemon[1857]: time=2026-09-23T16:15:38.151Z level=INFO msg="Uploading 1 narinfos"6358lb # [ 191.814603] niks3-server[1178]: 2026/09/23 16:15:38 INFO Signed narinfos id=68 count=16359lb # [ 191.823970] niks3-server[1178]: 2026/09/23 16:15:38 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6360worker1 # [ 191.846481] nix-grpc-daemon[1857]: time=2026-09-23T16:15:38.185Z level=INFO msg="Upload complete. (242ms)"6361lb # [ 191.838603] niks3-server[1178]: 2026/09/23 16:15:38 INFO Received complete push request method=POST path=/api/pushes/68/complete6362lb # [ 191.850012] niks3-server[1178]: 2026/09/23 16:15:38 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6363worker1 # [ 191.856254] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=5126364lb # [ 191.862295] niks3-server[1178]: 2026/09/23 16:15:38 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6365worker1 # [ 191.867138] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626366worker1 # [ 191.874267] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6367worker1 # [ 191.879406] nix-daemon[856]: reaped child process 1950, status = succeeded6368worker1 # [ 191.892250] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6369worker1 # [ 193.119461] nix-daemon[856]: reaped child process 1959, status = succeeded6370worker1 # [ 193.224994] nix-daemon[856]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)6371worker1 # [ 193.252451] nix-daemon[856]: reaped child process 1978, status = succeeded6372lb # [ 193.249193] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received push request method=POST path=/api/pushes6373worker1 # [ 193.287227] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.625Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6374worker1 # [ 193.290728] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.627Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6375lb # [ 193.319052] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6376worker1 # [ 193.331418] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.669Z level=INFO msg="Uploading 1 narinfos"6377lb # [ 193.335520] niks3-server[1178]: 2026/09/23 16:15:39 INFO Signed narinfos id=69 count=16378lb # [ 193.344724] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6379lb # [ 193.356775] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6380lb # [ 193.368393] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received complete push request method=POST path=/api/pushes/69/complete6381worker1 # [ 193.375922] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.714Z level=INFO msg="Upload complete. (271ms)"6382lb # [ 193.374763] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6383lb # [ 193.382764] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6384worker2 # [ 193.400697] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:39Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16385worker1 # [ 193.389267] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=10 outputs=16386worker2 # [ 193.415235] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:39Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06387worker1 # [ 193.412391] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626388worker1 # [ 193.417931] nix-daemon[856]: reaped child process 1957, status = succeeded6389worker1 # [ 193.426169] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626390worker1 # [ 193.438742] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626391worker1 # [ 193.449903] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6392worker1 # [ 193.594839] nix-daemon[856]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6393worker1 # [ 193.616519] nix-daemon[856]: reaped child process 1985, status = succeeded6394lb # [ 193.618993] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received push request method=POST path=/api/pushes6395worker1 # [ 193.630563] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.968Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6396worker1 # [ 193.632144] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.968Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6397lb # [ 193.649278] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6398worker1 # [ 193.661263] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.999Z level=INFO msg="Uploading 1 narinfos"6399lb # [ 193.663681] niks3-server[1178]: 2026/09/23 16:15:39 INFO Signed narinfos id=70 count=16400lb # [ 193.672894] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6401lb # [ 193.688335] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6402lb # [ 193.691835] niks3-server[1178]: 2026/09/23 16:15:40 INFO Received complete push request method=POST path=/api/pushes/70/complete6403worker1 # [ 193.698463] nix-grpc-daemon[1857]: time=2026-09-23T16:15:40.037Z level=INFO msg="Upload complete. (235ms)"6404lb # [ 193.701205] niks3-server[1178]: 2026/09/23 16:15:40 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6405worker1 # [ 193.707339] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=5126406worker1 # [ 193.723751] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626407worker1 # [ 193.731367] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6408worker1 # [ 193.737254] nix-daemon[856]: reaped child process 1980, status = succeeded6409worker1 # [ 193.751962] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6410worker1 # [ 195.002794] nix-daemon[856]: reaped child process 1989, status = succeeded6411worker1 # [ 195.132759] nix-daemon[856]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)6412worker1 # [ 195.153464] nix-daemon[856]: reaped child process 2008, status = succeeded6413lb # [ 195.157027] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received push request method=POST path=/api/pushes6414worker1 # [ 195.180389] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.518Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6415worker1 # [ 195.184569] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.518Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6416lb # [ 195.207622] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6417worker1 # [ 195.234293] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.572Z level=INFO msg="Uploading 1 narinfos"6418lb # [ 195.222341] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6419lb # [ 195.237119] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6420lb # [ 195.246826] niks3-server[1178]: 2026/09/23 16:15:41 INFO Signed narinfos id=71 count=16421lb # [ 195.253320] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6422lb # [ 195.261526] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received complete push request method=POST path=/api/pushes/71/complete6423worker1 # [ 195.268546] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.607Z level=INFO msg="Upload complete. (305ms)"6424lb # [ 195.270427] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6425worker2 # [ 195.293847] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:41Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16426worker1 # [ 195.281269] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=11 outputs=16427worker2 # [ 195.308828] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:41Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06428worker1 # [ 195.304419] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626429worker1 # [ 195.311429] nix-daemon[856]: reaped child process 1987, status = succeeded6430worker1 # [ 195.315397] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626431worker1 # [ 195.328111] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626432worker1 # [ 195.339453] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6433worker1 # [ 195.484176] nix-daemon[856]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)6434worker1 # [ 195.505407] nix-daemon[856]: reaped child process 2015, status = succeeded6435lb # [ 195.502232] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received push request method=POST path=/api/pushes6436worker1 # [ 195.516876] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.855Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6437worker1 # [ 195.518432] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.855Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6438lb # [ 195.533081] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6439worker1 # [ 195.543837] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.882Z level=INFO msg="Uploading 1 narinfos"6440lb # [ 195.546069] niks3-server[1178]: 2026/09/23 16:15:41 INFO Signed narinfos id=72 count=16441lb # [ 195.558824] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6442lb # [ 195.569575] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6443lb # [ 195.584447] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received complete push request method=POST path=/api/pushes/72/complete6444lb # [ 195.594891] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6445worker1 # [ 195.603646] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.942Z level=INFO msg="Upload complete. (248ms)"6446worker1 # [ 195.613312] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=5126447worker1 # [ 195.630901] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626448worker1 # [ 195.638348] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6449worker1 # [ 195.651008] nix-daemon[856]: reaped child process 2010, status = succeeded6450worker1 # [ 195.659292] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6451worker1 # [ 196.944779] nix-daemon[856]: reaped child process 2019, status = succeeded6452worker1 # [ 197.039058] nix-daemon[856]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)6453worker1 # [ 197.064457] nix-daemon[856]: reaped child process 2038, status = succeeded6454lb # [ 197.059499] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received push request method=POST path=/api/pushes6455worker1 # [ 197.080499] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.415Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6456worker1 # [ 197.082520] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.415Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6457lb # [ 197.109311] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6458worker1 # [ 197.121818] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.460Z level=INFO msg="Uploading 1 narinfos"6459lb # [ 197.125303] niks3-server[1178]: 2026/09/23 16:15:43 INFO Signed narinfos id=73 count=16460lb # [ 197.133138] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6461lb # [ 197.143188] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6462lb # [ 197.155414] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6463lb # [ 197.159861] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received complete push request method=POST path=/api/pushes/73/complete6464worker1 # [ 197.167565] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.506Z level=INFO msg="Upload complete. (297ms)"6465lb # [ 197.168630] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6466worker2 # [ 197.192917] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:43Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16467worker1 # [ 197.180749] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=12 outputs=16468worker2 # [ 197.199846] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:43Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06469worker1 # [ 197.196646] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626470worker1 # [ 197.205061] nix-daemon[856]: reaped child process 2017, status = succeeded6471worker1 # [ 197.209284] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626472worker1 # [ 197.219845] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.558Z level=INFO msg="All 1 paths already cached"6473worker1 # [ 197.229399] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626474worker1 # [ 197.238211] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6475worker1 # [ 197.369947] nix-daemon[856]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)6476worker1 # [ 197.393500] nix-daemon[856]: reaped child process 2045, status = succeeded6477lb # [ 197.390320] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received push request method=POST path=/api/pushes6478worker1 # [ 197.406355] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.744Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6479worker1 # [ 197.407825] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.744Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6480lb # [ 197.427754] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign6481worker1 # [ 197.436930] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.775Z level=INFO msg="Uploading 1 narinfos"6482lb # [ 197.441728] niks3-server[1178]: 2026/09/23 16:15:43 INFO Signed narinfos id=74 count=16483lb # [ 197.449477] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6484lb # [ 197.457672] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6485worker1 # [ 197.472577] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.811Z level=INFO msg="Upload complete. (222ms)"6486lb # [ 197.472742] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received complete push request method=POST path=/api/pushes/74/complete6487lb # [ 197.477144] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6488worker1 # [ 197.484716] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=6246489worker1 # [ 197.496244] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626490worker1 # [ 197.502947] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6491worker1 # [ 197.507568] nix-daemon[856]: reaped child process 2040, status = succeeded6492worker1 # [ 197.517612] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6493worker1 # [ 198.612749] nix-daemon[856]: reaped child process 2049, status = succeeded6494worker1 # [ 198.700868] nix-daemon[856]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)6495worker1 # [ 198.723604] nix-daemon[856]: reaped child process 2068, status = succeeded6496lb # [ 198.725844] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received push request method=POST path=/api/pushes6497worker1 # [ 198.736387] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.074Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6498worker1 # [ 198.738057] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.074Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6499lb # [ 198.768053] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6500lb # [ 198.781319] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign6501worker1 # [ 198.788650] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.126Z level=INFO msg="Uploading 1 narinfos"6502lb # [ 198.790364] niks3-server[1178]: 2026/09/23 16:15:45 INFO Signed narinfos id=75 count=16503lb # [ 198.795753] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6504lb # [ 198.811031] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received complete push request method=POST path=/api/pushes/75/complete6505worker1 # [ 198.819260] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.157Z level=INFO msg="Upload complete. (290ms)"6506lb # [ 198.818908] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6507lb # [ 198.822862] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6508worker2 # [ 198.849804] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:45Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16509worker1 # [ 198.834092] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=13 outputs=16510worker1 # [ 198.852441] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626511worker1 # [ 198.860816] nix-daemon[856]: reaped child process 2047, status = succeeded6512worker1 # [ 198.869082] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626513worker1 # [ 198.881463] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.220Z level=INFO msg="All 1 paths already cached"6514worker1 # [ 198.890676] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626515worker1 # [ 198.905902] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6516worker1 # [ 199.045302] nix-daemon[856]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)6517worker1 # [ 199.063336] nix-daemon[856]: reaped child process 2075, status = succeeded6518lb # [ 199.067087] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received push request method=POST path=/api/pushes6519worker1 # [ 199.077356] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.415Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6520worker1 # [ 199.079345] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.415Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6521lb # [ 199.098048] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6522worker1 # [ 199.108106] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.446Z level=INFO msg="Uploading 1 narinfos"6523lb # [ 199.111745] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign6524lb # [ 199.121079] niks3-server[1178]: 2026/09/23 16:15:45 INFO Signed narinfos id=76 count=16525lb # [ 199.129813] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6526worker1 # [ 199.137848] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.476Z level=INFO msg="Upload complete. (219ms)"6527lb # [ 199.137912] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received complete push request method=POST path=/api/pushes/76/complete6528lb # [ 199.142902] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6529worker1 # [ 199.146524] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=6246530worker1 # [ 199.162844] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626531worker1 # [ 199.170461] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6532worker1 # [ 199.175421] nix-daemon[856]: reaped child process 2070, status = succeeded6533worker1 # [ 199.189280] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6534lb # [ 199.270151] envoy[895]: conn peer=[2001:db8:1::1]:34588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9697 tx=5215 ms=54166535worker2 # [ 198.858296] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:45Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06536worker1 # [ 200.249921] nix-daemon[856]: reaped child process 2079, status = succeeded6537worker1 # [ 200.372408] nix-daemon[856]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)6538worker1 # [ 200.391845] nix-daemon[856]: reaped child process 2098, status = succeeded6539lb # [ 200.394830] niks3-server[1178]: 2026/09/23 16:15:46 INFO Received push request method=POST path=/api/pushes6540worker1 # [ 200.407539] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.744Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6541worker1 # [ 200.410948] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.744Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6542lb # [ 200.438206] niks3-server[1178]: 2026/09/23 16:15:46 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6543worker1 # [ 200.450113] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.788Z level=INFO msg="Uploading 1 narinfos"6544lb # [ 200.448865] niks3-server[1178]: 2026/09/23 16:15:46 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6545lb # [ 200.460655] niks3-server[1178]: 2026/09/23 16:15:46 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign6546lb # [ 200.471129] niks3-server[1178]: 2026/09/23 16:15:46 INFO Signed narinfos id=77 count=16547worker1 # [ 200.479147] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.817Z level=INFO msg="Upload complete. (280ms)"6548lb # [ 200.478047] niks3-server[1178]: 2026/09/23 16:15:46 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6549lb # [ 200.481897] niks3-server[1178]: 2026/09/23 16:15:46 INFO Received complete push request method=POST path=/api/pushes/77/complete6550lb # [ 200.487936] niks3-server[1178]: 2026/09/23 16:15:46 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6551worker2 # [ 200.507776] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:46Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16552worker1 # [ 200.491675] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=14 outputs=16553worker2 # [ 200.517294] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:46Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06554worker1 # [ 200.512768] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626555worker1 # [ 200.520767] nix-daemon[856]: reaped child process 2077, status = succeeded6556worker1 # [ 200.528680] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626557worker1 # [ 200.541077] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.879Z level=INFO msg="All 1 paths already cached"6558worker1 # [ 200.550275] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626559worker1 # [ 200.563306] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6560worker1 # [ 200.702380] nix-daemon[856]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)6561worker1 # [ 200.721280] nix-daemon[856]: reaped child process 2105, status = succeeded6562lb # [ 200.725067] niks3-server[1178]: 2026/09/23 16:15:47 INFO Received push request method=POST path=/api/pushes6563worker1 # [ 200.735290] nix-grpc-daemon[1857]: time=2026-09-23T16:15:47.073Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6564worker1 # [ 200.736915] nix-grpc-daemon[1857]: time=2026-09-23T16:15:47.073Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6565lb # [ 200.756387] niks3-server[1178]: 2026/09/23 16:15:47 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign6566worker1 # [ 200.764130] nix-grpc-daemon[1857]: time=2026-09-23T16:15:47.102Z level=INFO msg="Uploading 1 narinfos"6567lb # [ 200.769838] niks3-server[1178]: 2026/09/23 16:15:47 INFO Signed narinfos id=78 count=16568lb # [ 200.777543] niks3-server[1178]: 2026/09/23 16:15:47 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6569lb # [ 200.786402] niks3-server[1178]: 2026/09/23 16:15:47 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6570lb # [ 200.801834] niks3-server[1178]: 2026/09/23 16:15:47 INFO Received complete push request method=POST path=/api/pushes/78/complete6571worker1 # [ 200.808110] nix-grpc-daemon[1857]: time=2026-09-23T16:15:47.146Z level=INFO msg="Upload complete. (232ms)"6572lb # [ 200.807793] niks3-server[1178]: 2026/09/23 16:15:47 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6573worker1 # [ 200.817643] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=6246574worker1 # [ 200.835254] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626575worker1 # [ 200.841804] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6576worker1 # [ 200.847344] nix-daemon[856]: reaped child process 2100, status = succeeded6577worker1 # [ 200.857898] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6578worker1 # [ 202.089085] nix-daemon[856]: reaped child process 2109, status = succeeded6579worker1 # [ 202.186129] nix-daemon[856]: accepted connection from pid 2125, user nix-grpc-daemon (trusted)6580worker1 # [ 202.203455] nix-daemon[856]: reaped child process 2128, status = succeeded6581lb # [ 202.211109] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received push request method=POST path=/api/pushes6582worker1 # [ 202.223872] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.561Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6583worker1 # [ 202.225568] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.561Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6584lb # [ 202.255666] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign6585worker1 # [ 202.265405] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.603Z level=INFO msg="Uploading 1 narinfos"6586lb # [ 202.268971] niks3-server[1178]: 2026/09/23 16:15:48 INFO Signed narinfos id=79 count=16587lb # [ 202.277230] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6588lb # [ 202.292121] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6589worker1 # [ 202.303174] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.641Z level=INFO msg="Upload complete. (233ms)"6590lb # [ 202.302881] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6591lb # [ 202.309833] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received complete push request method=POST path=/api/pushes/79/complete6592worker2 # [ 202.328255] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:48Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16593lb # [ 202.315615] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6594worker1 # [ 202.313080] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=15 outputs=16595worker1 # [ 202.329077] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626596worker2 # [ 202.342142] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:48Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06597worker1 # [ 202.339207] nix-daemon[856]: reaped child process 2107, status = succeeded6598worker1 # [ 202.348470] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626599worker1 # [ 202.361027] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.699Z level=INFO msg="All 2 paths already cached"6600worker1 # [ 202.370268] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626601worker1 # [ 202.380595] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6602worker1 # [ 202.513289] nix-daemon[856]: accepted connection from pid 2132, user nix-grpc-daemon (trusted)6603worker1 # [ 202.534820] nix-daemon[856]: reaped child process 2135, status = succeeded6604lb # [ 202.530543] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received push request method=POST path=/api/pushes6605worker1 # [ 202.541795] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.880Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6606worker1 # [ 202.543284] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.880Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6607lb # [ 202.568543] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign6608worker1 # [ 202.579220] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.917Z level=INFO msg="Uploading 1 narinfos"6609lb # [ 202.582187] niks3-server[1178]: 2026/09/23 16:15:48 INFO Signed narinfos id=80 count=16610lb # [ 202.589304] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6611lb # [ 202.601327] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6612worker1 # [ 202.608962] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.947Z level=INFO msg="Upload complete. (217ms)"6613lb # [ 202.607650] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received complete push request method=POST path=/api/pushes/80/complete6614lb # [ 202.611322] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6615worker1 # [ 202.620513] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=7526616worker1 # [ 202.628248] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626617worker1 # [ 202.641725] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6618worker1 # [ 202.644417] nix-daemon[856]: reaped child process 2130, status = succeeded6619worker1 # [ 202.656219] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6620worker1 # [ 203.565629] nix-daemon[856]: reaped child process 2139, status = succeeded6621worker1 # [ 203.634131] nix-daemon[856]: accepted connection from pid 2155, user nix-grpc-daemon (trusted)6622worker1 # [ 203.655461] nix-daemon[856]: reaped child process 2158, status = succeeded6623lb # [ 203.658434] niks3-server[1178]: 2026/09/23 16:15:49 INFO Received push request method=POST path=/api/pushes6624worker1 # [ 203.672433] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.009Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6625worker1 # [ 203.674119] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.009Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6626worker1 # [ 203.715148] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.053Z level=INFO msg="Uploading 1 narinfos"6627lb # [ 203.701126] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign6628lb # [ 203.720892] niks3-server[1178]: 2026/09/23 16:15:50 INFO Signed narinfos id=81 count=16629lb # [ 203.723980] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6630lb # [ 203.738856] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6631lb # [ 203.750294] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6632lb # [ 203.753732] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received complete push request method=POST path=/api/pushes/81/complete6633worker1 # [ 203.759048] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.097Z level=INFO msg="Upload complete. (292ms)"6634lb # [ 203.758357] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6635worker2 # [ 203.782954] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:50Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16636worker1 # [ 203.771711] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=16 outputs=16637worker1 # [ 203.792727] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626638worker1 # [ 203.800273] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626639worker1 # [ 203.805843] nix-daemon[856]: reaped child process 2137, status = succeeded6640worker1 # [ 203.808558] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.147Z level=INFO msg="All 2 paths already cached"6641worker1 # [ 203.816164] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626642worker1 # [ 203.826401] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6643worker1 # [ 203.962688] nix-daemon[856]: accepted connection from pid 2162, user nix-grpc-daemon (trusted)6644worker1 # [ 203.980610] nix-daemon[856]: reaped child process 2165, status = succeeded6645lb # [ 203.983594] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received push request method=POST path=/api/pushes6646worker1 # [ 203.994152] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.332Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6647worker1 # [ 203.995670] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.332Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6648lb # [ 204.015280] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6649lb # [ 204.027553] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign6650worker1 # [ 204.035066] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.373Z level=INFO msg="Uploading 1 narinfos"6651lb # [ 204.038993] niks3-server[1178]: 2026/09/23 16:15:50 INFO Signed narinfos id=82 count=16652lb # [ 204.044296] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6653lb # [ 204.056284] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received complete push request method=POST path=/api/pushes/82/complete6654worker1 # [ 204.068405] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.406Z level=INFO msg="Upload complete. (231ms)"6655lb # [ 204.067741] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6656worker1 # [ 204.078620] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=7526657worker1 # [ 204.094435] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626658worker1 # [ 204.101423] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6659worker1 # [ 204.107084] nix-daemon[856]: reaped child process 2160, status = succeeded6660worker1 # [ 204.122015] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6661worker1 # [ 205.370673] nix-daemon[856]: reaped child process 2169, status = succeeded6662worker1 # [ 205.482743] nix-daemon[856]: accepted connection from pid 2185, user nix-grpc-daemon (trusted)6663worker1 # [ 205.501816] nix-daemon[856]: reaped child process 2188, status = succeeded6664lb # [ 205.507926] niks3-server[1178]: 2026/09/23 16:15:51 INFO Received push request method=POST path=/api/pushes6665worker1 # [ 205.522657] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.859Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6666worker1 # [ 205.524788] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.859Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6667lb # [ 205.549570] niks3-server[1178]: 2026/09/23 16:15:51 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6668worker1 # [ 205.564304] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.902Z level=INFO msg="Uploading 1 narinfos"6669lb # [ 205.562969] niks3-server[1178]: 2026/09/23 16:15:51 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign6670lb # [ 205.574462] niks3-server[1178]: 2026/09/23 16:15:51 INFO Signed narinfos id=83 count=16671lb # [ 205.579277] niks3-server[1178]: 2026/09/23 16:15:51 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6672lb # [ 205.590172] niks3-server[1178]: 2026/09/23 16:15:51 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6673lb # [ 205.595506] niks3-server[1178]: 2026/09/23 16:15:51 INFO Received complete push request method=POST path=/api/pushes/83/complete6674lb # [ 205.599114] niks3-server[1178]: 2026/09/23 16:15:51 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6675worker1 # [ 205.600857] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.939Z level=INFO msg="Upload complete. (268ms)"6676worker2 # [ 205.624939] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:51Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16677worker1 # [ 205.617249] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=17 outputs=16678worker1 # [ 205.636913] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626679worker1 # [ 205.640664] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626680worker1 # [ 205.645460] nix-daemon[856]: reaped child process 2167, status = succeeded6681worker1 # [ 205.652300] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.990Z level=INFO msg="All 2 paths already cached"6682worker1 # [ 205.659667] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397626683worker1 # [ 205.668662] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6684worker1 # [ 205.801805] nix-daemon[856]: accepted connection from pid 2192, user nix-grpc-daemon (trusted)6685worker1 # [ 205.821786] nix-daemon[856]: reaped child process 2195, status = succeeded6686lb # [ 205.825351] niks3-server[1178]: 2026/09/23 16:15:52 INFO Received push request method=POST path=/api/pushes6687worker1 # [ 205.835964] nix-grpc-daemon[1857]: time=2026-09-23T16:15:52.174Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6688worker1 # [ 205.837668] nix-grpc-daemon[1857]: time=2026-09-23T16:15:52.174Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6689lb # [ 205.855315] niks3-server[1178]: 2026/09/23 16:15:52 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6690worker1 # [ 205.867366] nix-grpc-daemon[1857]: time=2026-09-23T16:15:52.205Z level=INFO msg="Uploading 1 narinfos"6691lb # [ 205.868632] niks3-server[1178]: 2026/09/23 16:15:52 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign6692lb # [ 205.879566] niks3-server[1178]: 2026/09/23 16:15:52 INFO Signed narinfos id=84 count=16693lb # [ 205.892667] niks3-server[1178]: 2026/09/23 16:15:52 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6694lb # [ 205.899596] niks3-server[1178]: 2026/09/23 16:15:52 INFO Received complete push request method=POST path=/api/pushes/84/complete6695worker1 # [ 205.906671] nix-grpc-daemon[1857]: time=2026-09-23T16:15:52.245Z level=INFO msg="Upload complete. (227ms)"6696lb # [ 205.906578] niks3-server[1178]: 2026/09/23 16:15:52 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6697worker1 # [ 205.915708] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=7526698worker1 # [ 205.932805] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397626699worker1 # [ 205.939079] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6700worker1 # [ 205.943985] nix-daemon[856]: reaped child process 2190, status = succeeded6701worker1 # [ 205.958661] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6702worker1 # [ 207.240638] nix-daemon[856]: reaped child process 2199, status = succeeded6703worker1 # [ 207.343221] nix-daemon[856]: accepted connection from pid 2215, user nix-grpc-daemon (trusted)6704worker1 # [ 207.362570] nix-daemon[856]: reaped child process 2218, status = succeeded6705lb # [ 207.369260] niks3-server[1178]: 2026/09/23 16:15:53 INFO Received push request method=POST path=/api/pushes6706worker1 # [ 207.384445] nix-grpc-daemon[1857]: time=2026-09-23T16:15:53.720Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6707worker1 # [ 207.386130] nix-grpc-daemon[1857]: time=2026-09-23T16:15:53.720Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6708lb # [ 207.416930] niks3-server[1178]: 2026/09/23 16:15:53 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6709worker1 # [ 207.432113] nix-grpc-daemon[1857]: time=2026-09-23T16:15:53.770Z level=INFO msg="Uploading 1 narinfos"6710lb # [ 207.432042] niks3-server[1178]: 2026/09/23 16:15:53 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign6711lb # [ 207.435667] niks3-server[1178]: 2026/09/23 16:15:53 INFO Signed narinfos id=85 count=16712lb # [ 207.447412] niks3-server[1178]: 2026/09/23 16:15:53 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6713lb # [ 207.459749] niks3-server[1178]: 2026/09/23 16:15:53 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6714worker1 # [ 207.467619] nix-grpc-daemon[1857]: time=2026-09-23T16:15:53.806Z level=INFO msg="Upload complete. (297ms)"6715lb # [ 207.467363] niks3-server[1178]: 2026/09/23 16:15:53 INFO Received complete push request method=POST path=/api/pushes/85/complete6716lb # [ 207.471326] niks3-server[1178]: 2026/09/23 16:15:53 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6717worker1 # [ 207.481286] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=18 outputs=16718worker1 # [ 207.500926] nix-daemon[856]: reaped child process 2197, status = succeeded6719client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.29 seconds)6720worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6721worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6722client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6723client # [ 207.685614] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6724client # Running as unit: gpu.service; invocation ID: cfb8fc73c3c44e09825ba57aa4dfa9b96725client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6726lb: must succeed: curl -sf localhost:9901/clusters6727lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6728lb: must succeed: curl -sf localhost:9901/clusters6729lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6730worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6731client # [ 207.895256] nix[1620]: don't know how to build these paths:6732client # [ 207.896419] nix[1620]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6733worker2 # [ 207.900644] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:583446734worker1 # [ 207.888543] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397626735worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6736worker2 # [ 207.925777] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:54Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06737client # [ 207.930550] nix[1620]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6738lb: must succeed: curl -sf localhost:9901/clusters6739lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6740lb: must succeed: curl -sf localhost:9901/clusters6741lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6742worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6743worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6744client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6745lb # [ 209.269968] envoy[895]: conn peer=[2001:db8:1::1]:52714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27213 tx=12359 ms=160186746client # [ 237.937133] nix[1620]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6747client # [ 237.953019] nix[1620]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6748client # [ 237.985228] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6749client # [ 237.993015] systemd[1]: gpu.service: Failed with result 'exit-code'.6750client # [ 237.997447] systemd[1]: gpu.service: Consumed 198ms CPU time over 30.305s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6751client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.09 seconds)6752client: must succeed: journalctl -u gpu -o cat6753client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6754lb: must succeed: curl -sf localhost:9901/clusters6755lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6756lb: must succeed: curl -sf localhost:9901/clusters6757lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6758worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6759worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6760(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.10 seconds)6761subtest: low disk drains a worker and builds go to the other6762worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6763lb # [ 239.273888] envoy[895]: conn peer=[2001:db8:1::1]:57428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300676764worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.22 seconds)6765worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6766worker1 # [ 240.174651] nix-grpc-daemon[1797]: ts=2026-09-23T16:16:26Z level=info event=unhealthy reason=min_free6767worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6768client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26769worker2 # [ 240.913838] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:583446770client # don't know how to build these paths:6771client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6772worker2 # [ 240.923610] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583446773worker2 # [ 240.951543] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06774worker2 # [ 240.960998] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16775worker2 # [ 240.969583] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446776worker2 # [ 240.977169] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583446777client # copying 1 paths to 'lb:50051'...6778client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6779worker2 # [ 240.989122] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583446780worker2 # [ 241.004414] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6781worker2 # [ 241.146965] nix-daemon[844]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6782worker2 # [ 241.162874] nix-daemon[844]: reaped child process 1876, status = succeeded6783lb # [ 241.155607] niks3-server[1178]: 2026/09/23 16:16:27 INFO Received push request method=POST path=/api/pushes6784worker2 # [ 241.183748] nix-grpc-daemon[1578]: time=2026-09-23T16:16:27.508Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6785worker2 # [ 241.185858] nix-grpc-daemon[1578]: time=2026-09-23T16:16:27.508Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6786lb # [ 241.199430] niks3-server[1178]: 2026/09/23 16:16:27 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6787lb # [ 241.225396] niks3-server[1178]: 2026/09/23 16:16:27 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign6788worker2 # [ 241.240800] nix-grpc-daemon[1578]: time=2026-09-23T16:16:27.565Z level=INFO msg="Uploading 1 narinfos"6789lb # [ 241.230930] niks3-server[1178]: 2026/09/23 16:16:27 INFO Signed narinfos id=86 count=16790lb # [ 241.237312] niks3-server[1178]: 2026/09/23 16:16:27 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6791lb # [ 241.244413] niks3-server[1178]: 2026/09/23 16:16:27 INFO Received complete push request method=POST path=/api/pushes/86/complete6792lb # [ 241.253636] niks3-server[1178]: 2026/09/23 16:16:27 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6793worker2 # [ 241.276708] nix-grpc-daemon[1578]: time=2026-09-23T16:16:27.601Z level=INFO msg="Upload complete. (257ms)"6794worker2 # [ 241.288365] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=5126795worker2 # [ 241.300577] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446796worker2 # [ 241.314514] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6797worker2 # [ 241.328402] nix-daemon[844]: reaped child process 1871, status = succeeded6798client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6799client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6800worker2 # [ 241.338831] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6801client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6802client # farm-a> LOG-farm-a-drain6803worker2 # [ 242.416876] nix-daemon[844]: reaped child process 1880, status = succeeded6804worker2 # [ 242.513077] nix-daemon[844]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)6805worker2 # [ 242.532174] nix-daemon[844]: reaped child process 1899, status = succeeded6806lb # [ 242.523000] niks3-server[1178]: 2026/09/23 16:16:28 INFO Received push request method=POST path=/api/pushes6807worker2 # [ 242.559370] nix-grpc-daemon[1578]: time=2026-09-23T16:16:28.882Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6808worker2 # [ 242.561426] nix-grpc-daemon[1578]: time=2026-09-23T16:16:28.883Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6809lb # [ 242.571111] niks3-server[1178]: 2026/09/23 16:16:28 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign6810worker2 # [ 242.594622] nix-grpc-daemon[1578]: time=2026-09-23T16:16:28.919Z level=INFO msg="Uploading 1 narinfos"6811lb # [ 242.582229] niks3-server[1178]: 2026/09/23 16:16:28 INFO Signed narinfos id=87 count=16812lb # [ 242.589103] niks3-server[1178]: 2026/09/23 16:16:28 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6813lb # [ 242.599840] niks3-server[1178]: 2026/09/23 16:16:28 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6814lb # [ 242.611613] niks3-server[1178]: 2026/09/23 16:16:28 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6815worker2 # [ 242.633732] nix-grpc-daemon[1578]: time=2026-09-23T16:16:28.958Z level=INFO msg="Upload complete. (282ms)"6816lb # [ 242.620323] niks3-server[1178]: 2026/09/23 16:16:28 INFO Received complete push request method=POST path=/api/pushes/87/complete6817lb # [ 242.623619] niks3-server[1178]: 2026/09/23 16:16:28 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6818worker2 # [ 242.645124] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=19 outputs=16819worker2 # [ 242.669055] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:28Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06820worker2 # [ 242.678711] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:28Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16821worker2 # [ 242.684395] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446822worker2 # [ 242.689418] nix-daemon[844]: reaped child process 1878, status = succeeded6823worker2 # [ 242.692997] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583446824worker2 # [ 242.705430] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.030Z level=INFO msg="All 1 paths already cached"6825client # copying 1 paths to 'lb:50051'...6826client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6827worker2 # [ 242.715364] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583446828worker2 # [ 242.727412] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6829worker2 # [ 242.869012] nix-daemon[844]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6830worker2 # [ 242.888653] nix-daemon[844]: reaped child process 1908, status = succeeded6831lb # [ 242.877281] niks3-server[1178]: 2026/09/23 16:16:29 INFO Received push request method=POST path=/api/pushes6832worker2 # [ 242.902672] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.227Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6833worker2 # [ 242.904748] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.229Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6834lb # [ 242.911115] niks3-server[1178]: 2026/09/23 16:16:29 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6835worker2 # [ 242.937512] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.262Z level=INFO msg="Uploading 1 narinfos"6836lb # [ 242.925691] niks3-server[1178]: 2026/09/23 16:16:29 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign6837lb # [ 242.933939] niks3-server[1178]: 2026/09/23 16:16:29 INFO Signed narinfos id=88 count=16838lb # [ 242.939139] niks3-server[1178]: 2026/09/23 16:16:29 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6839lb # [ 242.947312] niks3-server[1178]: 2026/09/23 16:16:29 INFO Received complete push request method=POST path=/api/pushes/88/complete6840worker2 # [ 242.970460] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.295Z level=INFO msg="Upload complete. (229ms)"6841lb # [ 242.954929] niks3-server[1178]: 2026/09/23 16:16:29 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6842worker2 # [ 242.981075] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=6326843worker2 # [ 242.995090] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446844worker2 # [ 243.001588] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6845worker2 # [ 243.006675] nix-daemon[844]: reaped child process 1903, status = succeeded6846client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6847worker2 # [ 243.017143] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6848client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6849client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6850client # farm-b> LOG-farm-b-drain6851worker2 # [ 243.881881] nix-daemon[844]: reaped child process 1912, status = succeeded6852worker2 # [ 244.017765] nix-daemon[844]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)6853worker2 # [ 244.034926] nix-daemon[844]: reaped child process 1931, status = succeeded6854lb # [ 244.027391] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received push request method=POST path=/api/pushes6855worker2 # [ 244.056485] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6856worker2 # [ 244.058164] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.378Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6857worker2 # [ 244.102064] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.426Z level=INFO msg="Uploading 1 narinfos"6858lb # [ 244.075325] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign6859lb # [ 244.091385] niks3-server[1178]: 2026/09/23 16:16:30 INFO Signed narinfos id=89 count=16860lb # [ 244.098689] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6861lb # [ 244.108677] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6862lb # [ 244.121951] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6863lb # [ 244.127666] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received complete push request method=POST path=/api/pushes/89/complete6864worker2 # [ 244.143833] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.468Z level=INFO msg="Upload complete. (314ms)"6865lb # [ 244.131898] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6866worker2 # [ 244.161027] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=20 outputs=16867worker2 # [ 244.184472] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06868worker2 # [ 244.191762] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16869worker2 # [ 244.199353] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446870worker2 # [ 244.205717] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583446871worker2 # [ 244.211589] nix-daemon[844]: reaped child process 1910, status = succeeded6872worker2 # [ 244.215823] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.540Z level=INFO msg="All 2 paths already cached"6873client # copying 1 paths to 'lb:50051'...6874client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6875worker2 # [ 244.226430] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583446876worker2 # [ 244.240390] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6877worker2 # [ 244.381348] nix-daemon[844]: accepted connection from pid 1937, user nix-grpc-daemon (trusted)6878worker2 # [ 244.400466] nix-daemon[844]: reaped child process 1940, status = succeeded6879lb # [ 244.388902] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received push request method=POST path=/api/pushes6880worker2 # [ 244.413802] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.738Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6881worker2 # [ 244.415349] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.738Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6882lb # [ 244.419327] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign6883worker2 # [ 244.442474] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.767Z level=INFO msg="Uploading 1 narinfos"6884lb # [ 244.432643] niks3-server[1178]: 2026/09/23 16:16:30 INFO Signed narinfos id=90 count=16885lb # [ 244.435834] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6886lb # [ 244.450155] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6887lb # [ 244.462251] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received complete push request method=POST path=/api/pushes/90/complete6888worker2 # [ 244.483952] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.809Z level=INFO msg="Upload complete. (231ms)"6889lb # [ 244.468880] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6890worker2 # [ 244.496340] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=7606891worker2 # [ 244.510951] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583446892worker2 # [ 244.518151] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6893worker2 # [ 244.523141] nix-daemon[844]: reaped child process 1935, status = succeeded6894client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6895worker2 # [ 244.536954] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6896client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6897client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6898client # farm-top> LOG-farm-top-drain6899worker2 # [ 245.812546] nix-daemon[844]: reaped child process 1944, status = succeeded6900worker2 # [ 245.918086] nix-daemon[844]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)6901worker2 # [ 245.939758] nix-daemon[844]: reaped child process 1963, status = succeeded6902lb # [ 245.928621] niks3-server[1178]: 2026/09/23 16:16:32 INFO Received push request method=POST path=/api/pushes6903worker2 # [ 245.955210] nix-grpc-daemon[1578]: time=2026-09-23T16:16:32.278Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6904worker2 # [ 245.956954] nix-grpc-daemon[1578]: time=2026-09-23T16:16:32.279Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6905lb # [ 245.960202] niks3-server[1178]: 2026/09/23 16:16:32 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6906lb # [ 245.980535] niks3-server[1178]: 2026/09/23 16:16:32 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6907worker2 # [ 246.009098] nix-grpc-daemon[1578]: time=2026-09-23T16:16:32.333Z level=INFO msg="Uploading 1 narinfos"6908lb # [ 245.994771] niks3-server[1178]: 2026/09/23 16:16:32 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign6909lb # [ 246.004759] niks3-server[1178]: 2026/09/23 16:16:32 INFO Signed narinfos id=91 count=16910lb # [ 246.011063] niks3-server[1178]: 2026/09/23 16:16:32 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6911lb # [ 246.017866] niks3-server[1178]: 2026/09/23 16:16:32 INFO Received complete push request method=POST path=/api/pushes/91/complete6912lb # [ 246.028106] niks3-server[1178]: 2026/09/23 16:16:32 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6913worker2 # [ 246.044225] nix-grpc-daemon[1578]: time=2026-09-23T16:16:32.369Z level=INFO msg="Upload complete. (296ms)"6914worker2 # [ 246.057298] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=21 outputs=16915worker2 # [ 246.076572] nix-daemon[844]: reaped child process 1942, status = succeeded6916client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.41 seconds)6917client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath6918client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)6919worker1: must succeed: rm /nix/.rw-store/fill6920worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)6921worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6922worker1 # [ 247.185846] nix-grpc-daemon[1797]: ts=2026-09-23T16:16:33Z level=info event=healthy reason=min_free6923worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)6924(finished: subtest: low disk drains a worker and builds go to the other, in 9.26 seconds)6925(finished: run the VM test script, in 248.74 seconds)6926test script finished in 248.98s6927cleanup6928kill QemuMachine (pid 46)6929client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)6930client # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6931client # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6932client # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6933kill QemuMachine (pid 48)6934lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)6935lb # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6936lb # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6937lb # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6938kill QemuMachine (pid 51)6939worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)6940worker1 # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6941worker1 # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6942worker1 # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down6943kill QemuMachine (pid 52)6944worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)6945worker2 # [2026-09-23T16:16:35Z INFO virtiofsd] Client disconnected, shutting down6946worker2 # [2026-09-23T16:16:35Z INFO virtiofsd] Client disconnected, shutting down6947worker2 # [2026-09-23T16:16:35Z INFO virtiofsd] Client disconnected, shutting down6948(finished: cleanup, in 1.20 seconds)6949additionally exposed symbols:6950 client, lb, worker1, worker2,6951 vlan1,6952 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh