vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #519
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13worker2 # Disk image does not exist, creating the virtualisation disk image...14client # Formatting '/build/vm-state-client/tmp.89eV63jx49', fmt=raw size=107374182415client # mke2fs 1.47.4 (6-Mar-2025)16worker2: QEMU running (pid 51)17worker2 # Formatting '/build/vm-state-worker2/tmp.EvP4wNsNzk', fmt=raw size=107374182418client: QEMU running (pid 49)19worker1 # Disk image does not exist, creating the virtualisation disk image...20worker2 # mke2fs 1.47.4 (6-Mar-2025)21lb # Disk image does not exist, creating the virtualisation disk image...22lb: QEMU running (pid 50)23worker2 # Discarding device blocks: 0/262144 done24lb # Formatting '/build/vm-state-lb/tmp.zUFWG6t6bZ', fmt=raw size=107374182425client # Discarding device blocks: 0/262144 done26worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes27worker2 # Filesystem UUID: 218d393b-2fac-4c4a-8d56-08b2222e1a8228client # Creating filesystem with 262144 4k blocks and 65536 inodes29worker1: QEMU running (pid 52)30worker1 # Formatting '/build/vm-state-worker1/tmp.ilbWMUSjL6', fmt=raw size=107374182431client # Filesystem UUID: 386ec444-c406-47a2-b2dd-0117e896c25632lb # mke2fs 1.47.4 (6-Mar-2025)33worker2 # Superblock backups stored on blocks:34worker1 # mke2fs 1.47.4 (6-Mar-2025)35client # Superblock backups stored on blocks:36lb # Discarding device blocks: 0/262144 done37worker1 # Discarding device blocks: 0/262144 done38client # 32768, 98304, 163840, 22937639lb # Creating filesystem with 262144 4k blocks and 65536 inodes40worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes41client # 42lb # Filesystem UUID: c0390e1f-1753-470d-be95-7d18f0ff2ba343worker1 # Filesystem UUID: f896596c-c5ce-4568-8087-8dcfd162026c44client # Allocating group tables: 0/8 done45lb # Superblock backups stored on blocks:46worker1 # Superblock backups stored on blocks:47client # Writing inode tables: 0/8 done48lb # 32768, 98304, 163840, 22937649worker1 # 32768, 98304, 163840, 22937650client # Creating journal (8192 blocks): done51lb # 52worker1 # 53client # Writing superblocks and filesystem accounting information: 0/8 done54lb # Allocating group tables: 0/8 done55worker1 # Allocating group tables: 0/8 done56client # 57lb # Writing inode tables: 0/8 done58worker1 # Writing inode tables: 0/8 done59client # Virtualisation disk image created.60lb # Creating journal (8192 blocks): done61worker1 # Creating journal (8192 blocks): done62client # Starting virtiofs daemons...63lb # Writing superblocks and filesystem accounting information: 0/8 done64worker1 # Writing superblocks and filesystem accounting information: 0/8 done65client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)66lb # 67worker1 # 68client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether69lb # Virtualisation disk image created.70worker1 # Virtualisation disk image created.71client # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...72lb # Starting virtiofs daemons...73worker1 # Starting virtiofs daemons...74client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether80client # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...81lb # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...82worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...83client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether88worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89client # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...90lb # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...91worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...92client # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests93lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95client # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests96lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether97worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether98client # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests99lb # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...100worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...101worker2 # 32768, 98304, 163840, 229376102(finished: start all VMs, in 0.51 seconds)103lb # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests104worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests105worker2 # 106client: waiting for unit network-addresses-eth1.service107lb # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests108worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests109worker2 # Allocating group tables: 0/8 done110client: waiting for the VM to finish booting111lb # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests112worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests113worker2 # Writing inode tables: 0/8 done114worker2 # Creating journal (8192 blocks): done115worker2 # Writing superblocks and filesystem accounting information: 0/8 done116worker2 # 117worker2 # Virtualisation disk image created.118worker2 # Starting virtiofs daemons...119worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests129worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests130worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]133client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026134client # [ 0.000000] KASLR enabled135client # [ 0.000000] random: crng init done136client # [ 0.000000] Machine model: linux,dummy-virt137worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026138worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]139client # [ 0.000000] efi: UEFI not found.140worker1 # [ 0.000000] KASLR enabled141worker1 # [ 0.000000] random: crng init done142client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT143worker1 # [ 0.000000] Machine model: linux,dummy-virt144worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026145worker1 # [ 0.000000] efi: UEFI not found.146worker2 # [ 0.000000] KASLR enabled147client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]148worker2 # [ 0.000000] random: crng init done149worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT150client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]151worker2 # [ 0.000000] Machine model: linux,dummy-virt152client # [ 0.000000] Zone ranges:153worker2 # [ 0.000000] efi: UEFI not found.154worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]155client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]156worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]157worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT158client # [ 0.000000] DMA32 empty159worker1 # [ 0.000000] Zone ranges:160client # [ 0.000000] Normal empty161client # [ 0.000000] Device empty162worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]163worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]164client # [ 0.000000] Movable zone start for each node165worker1 # [ 0.000000] DMA32 empty166worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]167worker1 # [ 0.000000] Normal empty168client # [ 0.000000] Early memory node ranges169worker2 # [ 0.000000] Zone ranges:170worker1 # [ 0.000000] Device empty171client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]172worker1 # [ 0.000000] Movable zone start for each node173worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]174worker2 # [ 0.000000] DMA32 empty175worker1 # [ 0.000000] Early memory node ranges176worker2 # [ 0.000000] Normal empty177client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]178worker2 # [ 0.000000] Device empty179worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]180client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000181worker2 # [ 0.000000] Movable zone start for each node182worker2 # [ 0.000000] Early memory node ranges183worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]184client # [ 0.000000] psci: probing for conduit method from DT.185client # [ 0.000000] psci: PSCIv1.3 detected in firmware.186worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000187worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]188worker1 # [ 0.000000] psci: probing for conduit method from DT.189client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs190worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]191worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.192client # [ 0.000000] psci: Trusted OS migration not required193worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000194client # [ 0.000000] psci: SMC Calling Convention v1.1195worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs196worker2 # [ 0.000000] psci: probing for conduit method from DT.197worker1 # [ 0.000000] psci: Trusted OS migration not required198worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.199client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)200worker1 # [ 0.000000] psci: SMC Calling Convention v1.1201worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs202client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296203worker2 # [ 0.000000] psci: Trusted OS migration not required204worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)205client # [ 0.000000] Detected PIPT I-cache on CPU0206worker2 # [ 0.000000] psci: SMC Calling Convention v1.1207worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296208client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)209worker1 # [ 0.000000] Detected PIPT I-cache on CPU0210worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)211client # [ 0.000000] CPU features: detected: GICv3 CPU interface212worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)213client # [ 0.000000] CPU features: detected: Spectre-v4214worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296215client # [ 0.000000] CPU features: detected: Spectre-BHB216worker2 # [ 0.000000] Detected PIPT I-cache on CPU0217worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface218worker1 # [ 0.000000] CPU features: detected: Spectre-v4219client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38220worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)221worker1 # [ 0.000000] CPU features: detected: Spectre-BHB222client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23223worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface224worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38225worker2 # [ 0.000000] CPU features: detected: Spectre-v4226client # [ 0.000000] alternatives: applying boot alternatives227worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23228worker2 # [ 0.000000] CPU features: detected: Spectre-BHB229worker1 # [ 0.000000] alternatives: applying boot alternatives230worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38231worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23232worker2 # [ 0.000000] alternatives: applying boot alternatives233client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test/init regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration console=ttyAMA0,115200n8 console=tty0234worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0hxczffswy9q3fzf9cy2bm1icq25s5b8-nixos-system-worker1-test/init regInfo=/nix/store/gxyq0wqq5b5k7nykh0671gkp38b0z5pq-closure-info/registration console=ttyAMA0,115200n8 console=tty0235client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration", will be passed to user space.236worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gjdzrza82g2nqw3dgr89lmwyjzkwq204-nixos-system-worker2-test/init regInfo=/nix/store/6zp3n5xwb020l9g406n4c13fq3abx4ma-closure-info/registration console=ttyAMA0,115200n8 console=tty0237client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes238worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/gxyq0wqq5b5k7nykh0671gkp38b0z5pq-closure-info/registration", will be passed to user space.239client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)240worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes241worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/6zp3n5xwb020l9g406n4c13fq3abx4ma-closure-info/registration", will be passed to user space.242client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)243worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)244worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes245client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB246worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)247client # [ 0.000000] software IO TLB: area num 1.248worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)249worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB250client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB251worker1 # [ 0.000000] software IO TLB: area num 1.252worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)253worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB254client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)255worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB256client # [ 0.000000] Fallback order for Node 0: 0257worker2 # [ 0.000000] software IO TLB: area num 1.258worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)259client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216260worker1 # [ 0.000000] Fallback order for Node 0: 0261worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB262client # [ 0.000000] Policy zone: DMA263worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216264worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)265client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off266worker1 # [ 0.000000] Policy zone: DMA267worker2 # [ 0.000000] Fallback order for Node 0: 0268client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1269worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off270worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216271client # [ 0.000000] allocated 3145728 bytes of page_ext272worker2 # [ 0.000000] Policy zone: DMA273worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1274client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages275worker1 # [ 0.000000] allocated 3145728 bytes of page_ext276worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off277client # [ 0.000000] ftrace: allocated 294 pages with 4 groups278worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages279worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1280client # [ 0.000000] rcu: Hierarchical RCU implementation.281worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups282worker2 # [ 0.000000] allocated 3145728 bytes of page_ext283client # [ 0.000000] rcu: RCU event tracing is enabled.284worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.285worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages286client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.287worker1 # [ 0.000000] rcu: RCU event tracing is enabled.288worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups289client # [ 0.000000] Trampoline variant of Tasks RCU enabled.290worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.291worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.292client # [ 0.000000] Rude variant of Tasks RCU enabled.293worker2 # [ 0.000000] rcu: RCU event tracing is enabled.294worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.295client # [ 0.000000] Tracing variant of Tasks RCU enabled.296worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.297worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.298worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.299client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.300worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.301lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]302worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.303worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.304client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1305worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.306lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026307worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1308client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.309lb # [ 0.000000] KASLR enabled310worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.311lb # [ 0.000000] random: crng init done312worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.313lb # [ 0.000000] Machine model: linux,dummy-virt314worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1315client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.316lb # [ 0.000000] efi: UEFI not found.317worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.318worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.319client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.320lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT321client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0322worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.323worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.324lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]325client # [ 0.000000] GICv3: 256 SPIs implemented326worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0327lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]328client # [ 0.000000] GICv3: 0 Extended SPIs implemented329lb # [ 0.000000] Zone ranges:330worker1 # [ 0.000000] GICv3: 256 SPIs implemented331worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.332client # [ 0.000000] Root IRQ handler: gic_handle_irq333worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented334lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]335worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0336client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI337worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq338lb # [ 0.000000] DMA32 empty339worker2 # [ 0.000000] GICv3: 256 SPIs implemented340lb # [ 0.000000] Normal empty341client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0342worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI343worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented344lb # [ 0.000000] Device empty345worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0346client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000347lb # [ 0.000000] Movable zone start for each node348worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq349lb # [ 0.000000] Early memory node ranges350client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]351worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI352worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000353worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0354lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]355worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]356client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)357worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000358lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]359worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]360worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)361lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000362client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)363lb # [ 0.000000] psci: probing for conduit method from DT.364worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)365client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000366worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)367lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.368worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000369client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000370lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs371worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)372lb # [ 0.000000] psci: Trusted OS migration not required373worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000374client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.375worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000376lb # [ 0.000000] psci: SMC Calling Convention v1.1377worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.378worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000379lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)380client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns381worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.382worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns383client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).384lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296385lb # [ 0.000000] Detected PIPT I-cache on CPU0386worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).387worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns388client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns389lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)390worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).391worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns392lb # [ 0.000000] CPU features: detected: GICv3 CPU interface393client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns394client # [ 0.000033] arm-pv: using stolen time PV395lb # [ 0.000000] CPU features: detected: Spectre-v4396worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns397worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns398lb # [ 0.000000] CPU features: detected: Spectre-BHB399worker1 # [ 0.000033] arm-pv: using stolen time PV400worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns401client # [ 0.000490] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)402lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38403worker2 # [ 0.000030] arm-pv: using stolen time PV404client # [ 0.000687] Console: colour dummy device 80x25405worker1 # [ 0.000567] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)406lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23407client # [ 0.000694] printk: legacy console [tty0] enabled408worker1 # [ 0.000772] Console: colour dummy device 80x25409lb # [ 0.000000] alternatives: applying boot alternatives410worker2 # [ 0.000468] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)411worker1 # [ 0.000779] printk: legacy console [tty0] enabled412worker2 # [ 0.000678] Console: colour dummy device 80x25413client # [ 0.000916] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)414worker2 # [ 0.000686] printk: legacy console [tty0] enabled415client # [ 0.000923] pid_max: default: 32768 minimum: 301416worker1 # [ 0.000966] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)417worker1 # [ 0.000973] pid_max: default: 32768 minimum: 301418client # [ 0.000999] LSM: initializing lsm=capability,landlock,yama,bpf,ima419worker2 # [ 0.000889] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)420client # [ 0.001179] landlock: Up and running.421worker1 # [ 0.001049] LSM: initializing lsm=capability,landlock,yama,bpf,ima422worker2 # [ 0.000896] pid_max: default: 32768 minimum: 301423client # [ 0.001182] Yama: becoming mindful.424worker1 # [ 0.001184] landlock: Up and running.425lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test/init regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration console=ttyAMA0,115200n8 console=tty0426client # [ 0.001784] LSM support for eBPF active427worker1 # [ 0.001186] Yama: becoming mindful.428worker2 # [ 0.000983] LSM: initializing lsm=capability,landlock,yama,bpf,ima429worker1 # [ 0.001683] LSM support for eBPF active430worker2 # [ 0.001164] landlock: Up and running.431client # [ 0.001951] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)432worker2 # [ 0.001167] Yama: becoming mindful.433worker1 # [ 0.001859] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)434worker2 # [ 0.001777] LSM support for eBPF active435lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration", will be passed to user space.436client # [ 0.001984] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)437worker1 # [ 0.001895] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)438client # [ 0.003858] rcu: Hierarchical SRCU implementation.439worker2 # [ 0.001921] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)440lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes441worker1 # [ 0.003904] rcu: Hierarchical SRCU implementation.442client # [ 0.003864] rcu: Max phase no-delay instances is 1000.443worker2 # [ 0.001954] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)444lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)445worker1 # [ 0.003910] rcu: Max phase no-delay instances is 1000.446client # [ 0.005197] fsl-mc MSI: its@8080000 domain created447worker2 # [ 0.003853] rcu: Hierarchical SRCU implementation.448worker1 # [ 0.005192] fsl-mc MSI: its@8080000 domain created449client # [ 0.005289] EFI services will not be available.450lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)451worker2 # [ 0.003859] rcu: Max phase no-delay instances is 1000.452worker1 # [ 0.005281] EFI services will not be available.453client # [ 0.005359] smp: Bringing up secondary CPUs ...454worker2 # [ 0.005136] fsl-mc MSI: its@8080000 domain created455lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB456client # [ 0.005367] smp: Brought up 1 node, 1 CPU457worker1 # [ 0.005374] smp: Bringing up secondary CPUs ...458lb # [ 0.000000] software IO TLB: area num 1.459worker2 # [ 0.005223] EFI services will not be available.460worker1 # [ 0.005382] smp: Brought up 1 node, 1 CPU461client # [ 0.005370] SMP: Total of 1 processors activated.462worker2 # [ 0.005292] smp: Bringing up secondary CPUs ...463worker1 # [ 0.005385] SMP: Total of 1 processors activated.464client # [ 0.005376] CPU: All CPU(s) started at EL1465lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB466worker2 # [ 0.005300] smp: Brought up 1 node, 1 CPU467worker1 # [ 0.005392] CPU: All CPU(s) started at EL1468client # [ 0.005387] CPU features: detected: Branch Target Identification469worker2 # [ 0.005303] SMP: Total of 1 processors activated.470lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)471worker1 # [ 0.005403] CPU features: detected: Branch Target Identification472client # [ 0.005392] CPU features: detected: ARMv8.4 Translation Table Level473worker2 # [ 0.005309] CPU: All CPU(s) started at EL1474lb # [ 0.000000] Fallback order for Node 0: 0475worker1 # [ 0.005407] CPU features: detected: ARMv8.4 Translation Table Level476worker2 # [ 0.005320] CPU features: detected: Branch Target Identification477lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216478client # [ 0.005395] CPU features: detected: Instruction cache invalidation not required for I/D coherence479worker2 # [ 0.005325] CPU features: detected: ARMv8.4 Translation Table Level480worker1 # [ 0.005410] CPU features: detected: Instruction cache invalidation not required for I/D coherence481lb # [ 0.000000] Policy zone: DMA482client # [ 0.005399] CPU features: detected: Data cache clean to the PoU not required for I/D coherence483worker1 # [ 0.005414] CPU features: detected: Data cache clean to the PoU not required for I/D coherence484worker2 # [ 0.005327] CPU features: detected: Instruction cache invalidation not required for I/D coherence485lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off486client # [ 0.005403] CPU features: detected: Common not Private translations487worker1 # [ 0.005418] CPU features: detected: Common not Private translations488client # [ 0.005406] CPU features: detected: CRC32 instructions489lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1490worker2 # [ 0.005331] CPU features: detected: Data cache clean to the PoU not required for I/D coherence491worker1 # [ 0.005421] CPU features: detected: CRC32 instructions492lb # [ 0.000000] allocated 3145728 bytes of page_ext493worker2 # [ 0.005335] CPU features: detected: Common not Private translations494client # [ 0.005409] CPU features: detected: Data cache clean to Point of Deep Persistence495lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages496worker1 # [ 0.005424] CPU features: detected: Data cache clean to Point of Deep Persistence497worker2 # [ 0.005338] CPU features: detected: CRC32 instructions498client # [ 0.005412] CPU features: detected: Data cache clean to Point of Persistence499lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups500worker1 # [ 0.005427] CPU features: detected: Data cache clean to Point of Persistence501worker2 # [ 0.005341] CPU features: detected: Data cache clean to Point of Deep Persistence502lb # [ 0.000000] rcu: Hierarchical RCU implementation.503client # [ 0.005415] CPU features: detected: Data independent timing control (DIT)504lb # [ 0.000000] rcu: RCU event tracing is enabled.505worker1 # [ 0.005430] CPU features: detected: Data independent timing control (DIT)506client # [ 0.005418] CPU features: detected: E0PD507worker2 # [ 0.005344] CPU features: detected: Data cache clean to Point of Persistence508worker1 # [ 0.005433] CPU features: detected: E0PD509lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.510client # [ 0.005421] CPU features: detected: Enhanced Counter Virtualization511worker2 # [ 0.005348] CPU features: detected: Data independent timing control (DIT)512worker1 # [ 0.005436] CPU features: detected: Enhanced Counter Virtualization513lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.514worker2 # [ 0.005351] CPU features: detected: E0PD515client # [ 0.005424] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)516lb # [ 0.000000] Rude variant of Tasks RCU enabled.517worker1 # [ 0.005439] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)518worker2 # [ 0.005353] CPU features: detected: Enhanced Counter Virtualization519client # [ 0.005428] CPU features: detected: Enhanced Virtualization Traps520lb # [ 0.000000] Tracing variant of Tasks RCU enabled.521worker1 # [ 0.005443] CPU features: detected: Enhanced Virtualization Traps522worker2 # [ 0.005356] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)523client # [ 0.005431] CPU features: detected: Fine Grained Traps524lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.525worker1 # [ 0.005445] CPU features: detected: Fine Grained Traps526worker2 # [ 0.005360] CPU features: detected: Enhanced Virtualization Traps527client # [ 0.005434] CPU features: detected: Generic authentication (architected QARMA5 algorithm)528lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1529worker2 # [ 0.005363] CPU features: detected: Fine Grained Traps530worker1 # [ 0.005449] CPU features: detected: Generic authentication (architected QARMA5 algorithm)531client # [ 0.005439] CPU features: detected: RCpc load-acquire (LDAPR)532worker1 # [ 0.005454] CPU features: detected: RCpc load-acquire (LDAPR)533lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.534worker2 # [ 0.005366] CPU features: detected: Generic authentication (architected QARMA5 algorithm)535client # [ 0.005442] CPU features: detected: LSE atomic instructions536worker1 # [ 0.005457] CPU features: detected: LSE atomic instructions537worker2 # [ 0.005371] CPU features: detected: RCpc load-acquire (LDAPR)538client # [ 0.005445] CPU features: detected: Privileged Access Never539lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.540worker1 # [ 0.005460] CPU features: detected: Privileged Access Never541client # [ 0.005448] CPU features: detected: PMUv3542worker2 # [ 0.005374] CPU features: detected: LSE atomic instructions543worker1 # [ 0.005462] CPU features: detected: PMUv3544client # [ 0.005451] CPU features: detected: RAS Extension Support545worker2 # [ 0.005377] CPU features: detected: Privileged Access Never546lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.547worker1 # [ 0.005465] CPU features: detected: RAS Extension Support548worker2 # [ 0.005379] CPU features: detected: PMUv3549client # [ 0.005454] CPU features: detected: RASv1p1 Extension Support550lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0551worker1 # [ 0.005468] CPU features: detected: RASv1p1 Extension Support552worker2 # [ 0.005382] CPU features: detected: RAS Extension Support553lb # [ 0.000000] GICv3: 256 SPIs implemented554client # [ 0.005456] CPU features: detected: Random Number Generator555worker1 # [ 0.005471] CPU features: detected: Random Number Generator556lb # [ 0.000000] GICv3: 0 Extended SPIs implemented557worker2 # [ 0.005385] CPU features: detected: RASv1p1 Extension Support558client # [ 0.005459] CPU features: detected: Speculation barrier (SB)559lb # [ 0.000000] Root IRQ handler: gic_handle_irq560worker1 # [ 0.005474] CPU features: detected: Speculation barrier (SB)561worker2 # [ 0.005387] CPU features: detected: Random Number Generator562client # [ 0.005462] CPU features: detected: Stage-2 Force Write-Back563lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI564worker1 # [ 0.005476] CPU features: detected: Stage-2 Force Write-Back565worker2 # [ 0.005390] CPU features: detected: Speculation barrier (SB)566lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0567client # [ 0.005465] CPU features: detected: TLB range maintenance instructions568worker1 # [ 0.005479] CPU features: detected: TLB range maintenance instructions569worker2 # [ 0.005393] CPU features: detected: Stage-2 Force Write-Back570lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000571client # [ 0.005470] CPU features: detected: Speculative Store Bypassing Safe (SSBS)572worker2 # [ 0.005395] CPU features: detected: TLB range maintenance instructions573worker1 # [ 0.005485] CPU features: detected: Speculative Store Bypassing Safe (SSBS)574lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]575client # [ 0.005508] alternatives: applying system-wide alternatives576worker1 # [ 0.005522] alternatives: applying system-wide alternatives577worker2 # [ 0.005400] CPU features: detected: Speculative Store Bypassing Safe (SSBS)578client # [ 0.008543] CPU features: detected: BBM Level 2 without TLB conflict abort579lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)580worker1 # [ 0.008600] CPU features: detected: BBM Level 2 without TLB conflict abort581worker2 # [ 0.005437] alternatives: applying system-wide alternatives582worker2 # [ 0.008383] CPU features: detected: BBM Level 2 without TLB conflict abort583lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)584client # [ 0.008729] Memory: 1405632K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)585lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000586worker1 # [ 0.008753] Memory: 1405652K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)587client # [ 0.009169] devtmpfs: initialized588worker1 # [ 0.009168] devtmpfs: initialized589lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000590worker2 # [ 0.008576] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)591client # [ 0.011023] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)592worker2 # [ 0.008976] devtmpfs: initialized593worker1 # [ 0.011058] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)594lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.595client # [ 0.011052] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).596worker2 # [ 0.010839] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)597worker1 # [ 0.011081] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).598client # [ 0.011240] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL599lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns600worker2 # [ 0.010862] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).601worker1 # [ 0.011273] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL602client # [ 0.011245] 0 pages in range for non-PLT usage603lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).604worker1 # [ 0.011278] 0 pages in range for non-PLT usage605worker2 # [ 0.011049] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL606client # [ 0.011246] 508272 pages in range for PLT usage607worker1 # [ 0.011279] 508272 pages in range for PLT usage608worker2 # [ 0.011053] 0 pages in range for non-PLT usage609client # [ 0.011355] pinctrl core: initialized pinctrl subsystem610worker1 # [ 0.011401] pinctrl core: initialized pinctrl subsystem611lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns612client # [ 0.012087] DMI not present or invalid.613worker2 # [ 0.011054] 508272 pages in range for PLT usage614worker1 # [ 0.012198] DMI not present or invalid.615worker2 # [ 0.011171] pinctrl core: initialized pinctrl subsystem616client # [ 0.015536] NET: Registered PF_NETLINK/PF_ROUTE protocol family617lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns618worker1 # [ 0.015629] NET: Registered PF_NETLINK/PF_ROUTE protocol family619worker2 # [ 0.011985] DMI not present or invalid.620lb # [ 0.000030] arm-pv: using stolen time PV621client # [ 0.017819] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations622worker2 # [ 0.015388] NET: Registered PF_NETLINK/PF_ROUTE protocol family623worker1 # [ 0.017918] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations624lb # [ 0.000391] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)625client # [ 0.018028] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations626worker2 # [ 0.017653] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations627worker1 # [ 0.018109] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations628lb # [ 0.000593] Console: colour dummy device 80x25629worker2 # [ 0.017846] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations630client # [ 0.018237] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations631lb # [ 0.000601] printk: legacy console [tty0] enabled632worker1 # [ 0.018315] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations633client # [ 0.018261] audit: initializing netlink subsys (disabled)634worker1 # [ 0.018339] audit: initializing netlink subsys (disabled)635worker2 # [ 0.018051] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations636lb # [ 0.000795] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)637client # [ 0.018898] thermal_sys: Registered thermal governor 'fair_share'638worker1 # [ 0.018979] thermal_sys: Registered thermal governor 'fair_share'639worker2 # [ 0.018075] audit: initializing netlink subsys (disabled)640lb # [ 0.000802] pid_max: default: 32768 minimum: 301641client # [ 0.018900] thermal_sys: Registered thermal governor 'bang_bang'642worker2 # [ 0.018647] thermal_sys: Registered thermal governor 'fair_share'643worker1 # [ 0.018981] thermal_sys: Registered thermal governor 'bang_bang'644lb # [ 0.000883] LSM: initializing lsm=capability,landlock,yama,bpf,ima645client # [ 0.018904] thermal_sys: Registered thermal governor 'step_wise'646worker2 # [ 0.018649] thermal_sys: Registered thermal governor 'bang_bang'647worker1 # [ 0.018985] thermal_sys: Registered thermal governor 'step_wise'648lb # [ 0.001063] landlock: Up and running.649lb # [ 0.001066] Yama: becoming mindful.650client # [ 0.018907] thermal_sys: Registered thermal governor 'user_space'651worker2 # [ 0.018653] thermal_sys: Registered thermal governor 'step_wise'652worker1 # [ 0.018988] thermal_sys: Registered thermal governor 'user_space'653lb # [ 0.001526] LSM support for eBPF active654worker2 # [ 0.018656] thermal_sys: Registered thermal governor 'user_space'655lb # [ 0.001677] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)656client # [ 0.018913] thermal_sys: Registered thermal governor 'power_allocator'657worker1 # [ 0.018994] thermal_sys: Registered thermal governor 'power_allocator'658lb # [ 0.001709] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)659worker2 # [ 0.018661] thermal_sys: Registered thermal governor 'power_allocator'660client # [ 0.018948] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1661worker1 # [ 0.019021] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1662lb # [ 0.003524] rcu: Hierarchical SRCU implementation.663client # [ 0.018956] cpuidle: using governor ladder664worker1 # [ 0.019029] cpuidle: using governor ladder665worker2 # [ 0.018694] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1666client # [ 0.018962] cpuidle: using governor menu667lb # [ 0.003530] rcu: Max phase no-delay instances is 1000.668worker1 # [ 0.019035] cpuidle: using governor menu669worker2 # [ 0.018702] cpuidle: using governor ladder670lb # [ 0.004777] fsl-mc MSI: its@8080000 domain created671worker2 # [ 0.018708] cpuidle: using governor menu672client # [ 0.019159] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.673worker1 # [ 0.019262] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.674lb # [ 0.004873] EFI services will not be available.675client # [ 0.019174] ASID allocator initialised with 65536 entries676worker2 # [ 0.018907] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.677worker1 # [ 0.019279] ASID allocator initialised with 65536 entries678lb # [ 0.004983] smp: Bringing up secondary CPUs ...679client # [ 0.020376] Serial: AMBA PL011 UART driver680lb # [ 0.004992] smp: Brought up 1 node, 1 CPU681worker1 # [ 0.020521] Serial: AMBA PL011 UART driver682worker2 # [ 0.018922] ASID allocator initialised with 65536 entries683lb # [ 0.004995] SMP: Total of 1 processors activated.684worker2 # [ 0.020139] Serial: AMBA PL011 UART driver685client # [ 0.025722] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1686lb # [ 0.005000] CPU: All CPU(s) started at EL1687worker1 # [ 0.025807] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1688client # [ 0.025859] printk: console [ttyAMA0] enabled689worker1 # [ 0.025975] printk: console [ttyAMA0] enabled690worker2 # [ 0.025456] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1691lb # [ 0.005010] CPU features: detected: Branch Target Identification692worker2 # [ 0.025597] printk: console [ttyAMA0] enabled693lb # [ 0.005015] CPU features: detected: ARMv8.4 Translation Table Level694lb # [ 0.005018] CPU features: detected: Instruction cache invalidation not required for I/D coherence695lb # [ 0.005021] CPU features: detected: Data cache clean to the PoU not required for I/D coherence696lb # [ 0.005025] CPU features: detected: Common not Private translations697lb # [ 0.005028] CPU features: detected: CRC32 instructions698lb # [ 0.005031] CPU features: detected: Data cache clean to Point of Deep Persistence699lb # [ 0.005034] CPU features: detected: Data cache clean to Point of Persistence700client # [ 0.144262] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages701lb # [ 0.005037] CPU features: detected: Data independent timing control (DIT)702worker1 # [ 0.143172] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages703lb # [ 0.005040] CPU features: detected: E0PD704worker2 # [ 0.141729] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages705client # [ 0.144281] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page706worker1 # [ 0.143192] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page707lb # [ 0.005043] CPU features: detected: Enhanced Counter Virtualization708worker2 # [ 0.141748] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page709client # [ 0.144287] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages710worker1 # [ 0.143197] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages711worker2 # [ 0.141754] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages712lb # [ 0.005046] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)713client # [ 0.144291] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page714worker1 # [ 0.143202] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page715lb # [ 0.005050] CPU features: detected: Enhanced Virtualization Traps716worker2 # [ 0.141759] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page717client # [ 0.144296] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages718worker1 # [ 0.143206] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages719lb # [ 0.005052] CPU features: detected: Fine Grained Traps720worker2 # [ 0.141763] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages721client # [ 0.144300] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page722worker1 # [ 0.143211] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page723lb # [ 0.005056] CPU features: detected: Generic authentication (architected QARMA5 algorithm)724worker2 # [ 0.141767] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page725client # [ 0.144304] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages726worker1 # [ 0.143215] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages727lb # [ 0.005060] CPU features: detected: RCpc load-acquire (LDAPR)728worker2 # [ 0.141772] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages729client # [ 0.144308] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page730worker1 # [ 0.143219] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page731lb # [ 0.005063] CPU features: detected: LSE atomic instructions732worker2 # [ 0.141776] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page733lb # [ 0.005066] CPU features: detected: Privileged Access Never734lb # [ 0.005068] CPU features: detected: PMUv3735lb # [ 0.005072] CPU features: detected: RAS Extension Support736client # [ 0.151699] fbcon: Taking over console737worker1 # [ 0.150627] fbcon: Taking over console738client # [ 0.151717] ACPI: Interpreter disabled.739lb # [ 0.005074] CPU features: detected: RASv1p1 Extension Support740worker2 # [ 0.149010] fbcon: Taking over console741worker1 # [ 0.150643] ACPI: Interpreter disabled.742worker2 # [ 0.149026] ACPI: Interpreter disabled.743lb # [ 0.005077] CPU features: detected: Random Number Generator744lb # [ 0.005080] CPU features: detected: Speculation barrier (SB)745lb # [ 0.005082] CPU features: detected: Stage-2 Force Write-Back746lb # [ 0.005085] CPU features: detected: TLB range maintenance instructions747lb # [ 0.005090] CPU features: detected: Speculative Store Bypassing Safe (SSBS)748lb # [ 0.005126] alternatives: applying system-wide alternatives749lb # [ 0.008077] CPU features: detected: BBM Level 2 without TLB conflict abort750lb # [ 0.008279] Memory: 1405640K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)751lb # [ 0.008641] devtmpfs: initialized752client # [ 0.160712] iommu: Default domain type: Translated753lb # [ 0.010454] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)754worker1 # [ 0.159663] iommu: Default domain type: Translated755worker2 # [ 0.158065] iommu: Default domain type: Translated756client # [ 0.160722] iommu: DMA domain TLB invalidation policy: strict mode757worker1 # [ 0.159675] iommu: DMA domain TLB invalidation policy: strict mode758lb # [ 0.010476] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).759client # [ 0.161116] SCSI subsystem initialized760worker2 # [ 0.158076] iommu: DMA domain TLB invalidation policy: strict mode761worker1 # [ 0.160051] SCSI subsystem initialized762worker2 # [ 0.158432] SCSI subsystem initialized763lb # [ 0.010667] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL764lb # [ 0.010672] 0 pages in range for non-PLT usage765client # [ 0.163088] usbcore: registered new interface driver usbfs766worker1 # [ 0.161997] usbcore: registered new interface driver usbfs767worker2 # [ 0.160358] usbcore: registered new interface driver usbfs768lb # [ 0.010673] 508272 pages in range for PLT usage769client # [ 0.163120] usbcore: registered new interface driver hub770worker1 # [ 0.162035] usbcore: registered new interface driver hub771lb # [ 0.010785] pinctrl core: initialized pinctrl subsystem772worker2 # [ 0.160390] usbcore: registered new interface driver hub773client # [ 0.163137] usbcore: registered new device driver usb774worker1 # [ 0.162051] usbcore: registered new device driver usb775lb # [ 0.011532] DMI not present or invalid.776worker2 # [ 0.160406] usbcore: registered new device driver usb777client # [ 0.163400] pps_core: LinuxPPS API ver. 1 registered778worker1 # [ 0.162315] pps_core: LinuxPPS API ver. 1 registered779lb # [ 0.014568] NET: Registered PF_NETLINK/PF_ROUTE protocol family780worker2 # [ 0.160687] pps_core: LinuxPPS API ver. 1 registered781client # [ 0.163406] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>782worker1 # [ 0.162321] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>783lb # [ 0.016858] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations784client # [ 0.163423] PTP clock support registered785worker2 # [ 0.160694] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>786worker1 # [ 0.162331] PTP clock support registered787client # [ 0.163473] EDAC MC: Ver: 3.0.0788worker2 # [ 0.160703] PTP clock support registered789worker1 # [ 0.162389] EDAC MC: Ver: 3.0.0790lb # [ 0.017033] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations791worker2 # [ 0.160753] EDAC MC: Ver: 3.0.0792client # [ 0.168047] scmi_core: SCMI protocol bus registered793worker1 # [ 0.166938] scmi_core: SCMI protocol bus registered794lb # [ 0.017237] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations795worker2 # [ 0.165204] scmi_core: SCMI protocol bus registered796lb # [ 0.017257] audit: initializing netlink subsys (disabled)797worker2 # [ 0.166149] FPGA manager framework798client # [ 0.169022] FPGA manager framework799worker1 # [ 0.167925] FPGA manager framework800client # [ 0.169949] vgaarb: loaded801lb # [ 0.018109] thermal_sys: Registered thermal governor 'fair_share'802worker1 # [ 0.168895] vgaarb: loaded803worker2 # [ 0.167089] vgaarb: loaded804lb # [ 0.018111] thermal_sys: Registered thermal governor 'bang_bang'805client # [ 0.170575] clocksource: Switched to clocksource arch_sys_counter806worker1 # [ 0.169529] clocksource: Switched to clocksource arch_sys_counter807lb # [ 0.018114] thermal_sys: Registered thermal governor 'step_wise'808worker2 # [ 0.167719] clocksource: Switched to clocksource arch_sys_counter809client # [ 0.176872] VFS: Disk quotas dquot_6.6.0810worker1 # [ 0.175851] VFS: Disk quotas dquot_6.6.0811worker2 # [ 0.174054] VFS: Disk quotas dquot_6.6.0812client # [ 0.176935] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)813worker1 # [ 0.175901] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)814worker2 # [ 0.174100] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)815client # [ 0.178668] netfs: FS-Cache loaded816worker2 # [ 0.175702] netfs: FS-Cache loaded817worker1 # [ 0.177558] netfs: FS-Cache loaded818client # [ 0.178804] pnp: PnP ACPI: disabled819worker1 # [ 0.177680] pnp: PnP ACPI: disabled820worker2 # [ 0.176256] pnp: PnP ACPI: disabled821lb # [ 0.018117] thermal_sys: Registered thermal governor 'user_space'822lb # [ 0.018121] thermal_sys: Registered thermal governor 'power_allocator'823lb # [ 0.018147] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1824lb # [ 0.018155] cpuidle: using governor ladder825client # [ 0.182558] NET: Registered PF_INET protocol family826lb # [ 0.018161] cpuidle: using governor menu827worker1 # [ 0.181515] NET: Registered PF_INET protocol family828worker2 # [ 0.179625] NET: Registered PF_INET protocol family829lb # [ 0.018346] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.830lb # [ 0.018361] ASID allocator initialised with 65536 entries831lb # [ 0.019569] Serial: AMBA PL011 UART driver832lb # [ 0.024866] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1833lb # [ 0.025029] printk: console [ttyAMA0] enabled834client # [ 0.183465] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)835worker1 # [ 0.182419] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)836worker2 # [ 0.180511] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)837lb # [ 0.144791] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages838lb # [ 0.144811] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page839lb # [ 0.144817] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages840lb # [ 0.144821] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page841lb # [ 0.144826] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages842lb # [ 0.144830] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page843lb # [ 0.144834] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages844lb # [ 0.144838] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page845lb # [ 0.152361] fbcon: Taking over console846lb # [ 0.152377] ACPI: Interpreter disabled.847lb # [ 0.154260] iommu: Default domain type: Translated848lb # [ 0.154270] iommu: DMA domain TLB invalidation policy: strict mode849lb # [ 0.156072] SCSI subsystem initialized850lb # [ 0.163172] usbcore: registered new interface driver usbfs851lb # [ 0.163205] usbcore: registered new interface driver hub852lb # [ 0.163222] usbcore: registered new device driver usb853lb # [ 0.163507] pps_core: LinuxPPS API ver. 1 registered854lb # [ 0.163513] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>855lb # [ 0.163530] PTP clock support registered856lb # [ 0.163580] EDAC MC: Ver: 3.0.0857lb # [ 0.168217] scmi_core: SCMI protocol bus registered858lb # [ 0.169197] FPGA manager framework859lb # [ 0.170197] vgaarb: loaded860lb # [ 0.170853] clocksource: Switched to clocksource arch_sys_counter861lb # [ 0.172888] VFS: Disk quotas dquot_6.6.0862lb # [ 0.172923] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)863lb # [ 0.176636] netfs: FS-Cache loaded864client # [ 0.212928] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)865lb # [ 0.176775] pnp: PnP ACPI: disabled866worker2 # [ 0.209920] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)867client # [ 0.212982] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)868worker2 # [ 0.209968] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)869client # [ 0.213016] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)870worker2 # [ 0.209996] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)871client # [ 0.213099] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)872worker2 # [ 0.210072] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)873client # [ 0.213242] TCP: Hash tables configured (established 16384 bind 16384)874worker2 # [ 0.210215] TCP: Hash tables configured (established 16384 bind 16384)875worker1 # [ 0.212147] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)876client # [ 0.213344] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)877worker2 # [ 0.210301] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)878worker1 # [ 0.212200] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)879client # [ 0.213381] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)880worker2 # [ 0.210339] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)881worker1 # [ 0.212230] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)882client # [ 0.213416] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)883worker2 # [ 0.210377] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)884lb # [ 0.180649] NET: Registered PF_INET protocol family885client # [ 0.213527] NET: Registered PF_UNIX/PF_LOCAL protocol family886worker1 # [ 0.212312] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)887lb # [ 0.180917] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)888worker2 # [ 0.210510] NET: Registered PF_UNIX/PF_LOCAL protocol family889client # [ 0.213554] NET: Registered PF_XDP protocol family890worker2 # [ 0.210529] NET: Registered PF_XDP protocol family891worker1 # [ 0.212455] TCP: Hash tables configured (established 16384 bind 16384)892client # [ 0.213572] PCI: CLS 0 bytes, default 64893worker2 # [ 0.210548] PCI: CLS 0 bytes, default 64894client # [ 0.213820] Trying to unpack rootfs image as initramfs...895worker1 # [ 0.212549] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)896worker2 # [ 0.210809] Trying to unpack rootfs image as initramfs...897worker1 # [ 0.212588] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)898worker1 # [ 0.212625] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)899worker1 # [ 0.212758] NET: Registered PF_UNIX/PF_LOCAL protocol family900worker1 # [ 0.212779] NET: Registered PF_XDP protocol family901worker1 # [ 0.212797] PCI: CLS 0 bytes, default 64902worker1 # [ 0.213054] Trying to unpack rootfs image as initramfs...903client # [ 0.228316] kvm [1]: HYP mode not available904worker2 # [ 0.225291] kvm [1]: HYP mode not available905worker1 # [ 0.231237] kvm [1]: HYP mode not available906lb # [ 0.211122] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907lb # [ 0.211172] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908lb # [ 0.211203] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909lb # [ 0.211279] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910lb # [ 0.211423] TCP: Hash tables configured (established 16384 bind 16384)911lb # [ 0.211509] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912lb # [ 0.211547] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913lb # [ 0.211583] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914lb # [ 0.211676] NET: Registered PF_UNIX/PF_LOCAL protocol family915lb # [ 0.211713] NET: Registered PF_XDP protocol family916lb # [ 0.211733] PCI: CLS 0 bytes, default 64917lb # [ 0.211969] Trying to unpack rootfs image as initramfs...918lb # [ 0.234888] kvm [1]: HYP mode not available919client # [ 0.335159] Initialise system trusted keyrings920worker2 # [ 0.332322] Initialise system trusted keyrings921client # [ 0.335915] workingset: timestamp_bits=42 max_order=19 bucket_order=0922worker2 # [ 0.333070] workingset: timestamp_bits=42 max_order=19 bucket_order=0923client # [ 0.337253] squashfs: version 4.0 (2009/01/31) Phillip Lougher924worker2 # [ 0.334428] squashfs: version 4.0 (2009/01/31) Phillip Lougher925worker2 # [ 0.335210] 9p: Installing v9fs 9p2000 file system support926worker1 # [ 0.343020] Initialise system trusted keyrings927worker1 # [ 0.343779] workingset: timestamp_bits=42 max_order=19 bucket_order=0928worker1 # [ 0.345049] squashfs: version 4.0 (2009/01/31) Phillip Lougher929client # [ 0.342645] 9p: Installing v9fs 9p2000 file system support930worker1 # [ 0.345900] 9p: Installing v9fs 9p2000 file system support931client # [ 0.363477] Key type asymmetric registered932client # [ 0.363504] Asymmetric key parser 'x509' registered933client # [ 0.363583] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)934client # [ 0.365785] io scheduler mq-deadline registered935client # [ 0.365797] io scheduler kyber registered936worker1 # [ 0.365893] Key type asymmetric registered937worker2 # [ 0.363955] Key type asymmetric registered938worker1 # [ 0.365922] Asymmetric key parser 'x509' registered939worker2 # [ 0.363984] Asymmetric key parser 'x509' registered940worker1 # [ 0.366008] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)941worker2 # [ 0.364059] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)942worker2 # [ 0.366256] io scheduler mq-deadline registered943worker2 # [ 0.366269] io scheduler kyber registered944worker1 # [ 0.373613] io scheduler mq-deadline registered945worker1 # [ 0.373638] io scheduler kyber registered946client # [ 0.378755] pl061_gpio 9030000.pl061: PL061 GPIO chip registered947worker2 # [ 0.375873] pl061_gpio 9030000.pl061: PL061 GPIO chip registered948worker2 # [ 0.377331] ledtrig-cpu: registered to indicate activity on CPUs949worker2 # [ 0.377745] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:950worker2 # [ 0.377763] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000951lb # [ 0.343388] Initialise system trusted keyrings952worker2 # [ 0.377775] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000953client # [ 0.380185] ledtrig-cpu: registered to indicate activity on CPUs954lb # [ 0.344137] workingset: timestamp_bits=42 max_order=19 bucket_order=0955worker2 # [ 0.377784] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000956client # [ 0.380580] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:957worker1 # [ 0.385713] pl061_gpio 9030000.pl061: PL061 GPIO chip registered958lb # [ 0.345402] squashfs: version 4.0 (2009/01/31) Phillip Lougher959worker2 # [ 0.377805] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits960client # [ 0.380596] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000961lb # [ 0.346146] 9p: Installing v9fs 9p2000 file system support962worker2 # [ 0.377832] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]963client # [ 0.380609] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000964worker2 # [ 0.377926] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00965client # [ 0.380617] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000966worker2 # [ 0.377937] pci_bus 0000:00: root bus resource [bus 00-ff]967worker2 # [ 0.377943] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]968client # [ 0.380655] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits969worker2 # [ 0.377949] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]970client # [ 0.380682] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]971worker2 # [ 0.377954] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]972client # [ 0.380757] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00973worker2 # [ 0.378042] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint974client # [ 0.380766] pci_bus 0000:00: root bus resource [bus 00-ff]975worker1 # [ 0.387118] ledtrig-cpu: registered to indicate activity on CPUs976client # [ 0.380772] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]977worker2 # [ 0.378480] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint978worker1 # [ 0.387543] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:979worker2 # [ 0.378663] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]980client # [ 0.380778] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]981worker2 # [ 0.378679] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]982worker1 # [ 0.387562] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000983client # [ 0.380783] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]984worker2 # [ 0.378709] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]985worker1 # [ 0.387574] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000986client # [ 0.380868] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint987worker2 # [ 0.378725] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]988client # [ 0.381327] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint989worker1 # [ 0.387582] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000990worker2 # [ 0.379184] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint991client # [ 0.381516] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]992worker2 # [ 0.379367] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]993worker1 # [ 0.387619] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits994client # [ 0.381533] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]995worker2 # [ 0.379383] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]996worker1 # [ 0.387647] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]997client # [ 0.381563] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998worker2 # [ 0.379413] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]999client # [ 0.381580] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1000worker1 # [ 0.387726] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001001worker1 # [ 0.387736] pci_bus 0000:00: root bus resource [bus 00-ff]1002client # [ 0.382046] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1003worker2 # [ 0.398895] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1004worker1 # [ 0.387743] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1005client # [ 0.382232] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1006worker2 # [ 0.399086] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1007worker1 # [ 0.387748] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1008client # [ 0.382248] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1009worker2 # [ 0.399102] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1010client # [ 0.382278] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1011worker1 # [ 0.387753] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1012worker2 # [ 0.399132] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1013client # [ 0.382767] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1014worker1 # [ 0.387809] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1015worker2 # [ 0.399586] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1016client # [ 0.382954] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1017worker1 # [ 0.388256] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1018client # [ 0.382970] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1019worker1 # [ 0.388444] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1020client # [ 0.382999] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1021worker1 # [ 0.388460] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1022lb # [ 0.366844] Key type asymmetric registered1023client # [ 0.383453] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1024worker1 # [ 0.388491] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025client # [ 0.383647] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1026worker1 # [ 0.388507] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1027client # [ 0.383663] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1028worker1 # [ 0.388973] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1029client # [ 0.383693] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1030worker1 # [ 0.389159] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1031client # [ 0.383713] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1032worker1 # [ 0.389175] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1033client # [ 0.384167] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1034worker1 # [ 0.389205] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035client # [ 0.384358] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1036worker1 # [ 0.389697] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1037client # [ 0.384389] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1038worker1 # [ 0.389886] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1039worker1 # [ 0.389902] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1040client # [ 0.384847] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1041client # [ 0.385049] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1042worker1 # [ 0.389932] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1043client # [ 0.385079] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044lb # [ 0.374920] Asymmetric key parser 'x509' registered1045worker1 # [ 0.390395] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1046worker1 # [ 0.390589] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1047lb # [ 0.375006] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1048client # [ 0.385488] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1049worker1 # [ 0.390603] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1050lb # [ 0.376616] io scheduler mq-deadline registered1051client # [ 0.385671] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1052lb # [ 0.376628] io scheduler kyber registered1053worker1 # [ 0.390633] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054client # [ 0.385939] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1055worker1 # [ 0.390652] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1056client # [ 0.386127] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1057worker1 # [ 0.391114] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1058client # [ 0.386157] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059worker1 # [ 0.391307] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1060worker1 # [ 0.391337] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1061worker2 # [ 0.411853] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1062worker1 # [ 0.391809] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1063worker2 # [ 0.411889] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1064worker1 # [ 0.392001] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1065worker2 # [ 0.411919] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1066worker1 # [ 0.392031] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067worker2 # [ 0.411938] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1068worker1 # [ 0.392475] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1069worker2 # [ 0.412469] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1070worker1 # [ 0.392658] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1071worker2 # [ 0.412662] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1072worker2 # [ 0.412691] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1073worker1 # [ 0.392912] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1074worker1 # [ 0.393100] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1075worker2 # [ 0.413146] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1076worker1 # [ 0.393130] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077lb # [ 0.387018] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1078worker2 # [ 0.413333] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1079client # [ 0.426768] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1080worker2 # [ 0.413362] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081client # [ 0.426976] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1082worker2 # [ 0.413770] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1083client # [ 0.427008] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1084worker2 # [ 0.413951] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1085lb # [ 0.388422] ledtrig-cpu: registered to indicate activity on CPUs1086client # [ 0.427552] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1087worker2 # [ 0.414238] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1088lb # [ 0.388813] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1089client # [ 0.427743] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1090worker2 # [ 0.414429] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1091client # [ 0.427774] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1092lb # [ 0.388831] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001093worker2 # [ 0.414460] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094client # [ 0.428242] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1095worker2 # [ 0.414913] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1096lb # [ 0.388842] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001097client # [ 0.428527] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1098worker2 # [ 0.415099] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1099lb # [ 0.388851] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001100client # [ 0.428545] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1101worker2 # [ 0.415129] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1102lb # [ 0.388874] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1103client # [ 0.428576] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104worker2 # [ 0.415583] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1105worker1 # [ 0.433655] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1106client # [ 0.429054] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1107worker2 # [ 0.415788] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1108lb # [ 0.388901] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1109client # [ 0.429240] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1110worker1 # [ 0.433868] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1111lb # [ 0.388988] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001112worker2 # [ 0.415818] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1113client # [ 0.429256] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1114worker1 # [ 0.433901] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115lb # [ 0.388997] pci_bus 0000:00: root bus resource [bus 00-ff]1116worker2 # [ 0.416286] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1117client # [ 0.429287] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118lb # [ 0.389003] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1119worker1 # [ 0.434421] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1120worker2 # [ 0.416582] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1121client # [ 0.429874] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1122lb # [ 0.389009] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1123worker1 # [ 0.434615] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1124worker2 # [ 0.416599] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1125client # [ 0.429886] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1126lb # [ 0.389013] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1127worker2 # [ 0.416628] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1128worker1 # [ 0.434645] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129client # [ 0.429891] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1130lb # [ 0.389070] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1131worker2 # [ 0.417097] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1132worker1 # [ 0.435112] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1133worker2 # [ 0.417281] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1134worker1 # [ 0.435415] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1135client # [ 0.429937] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1136lb # [ 0.389509] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1137worker2 # [ 0.417297] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1138worker1 # [ 0.435434] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1139lb # [ 0.389697] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1140client # [ 0.429986] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1141worker2 # [ 0.417326] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1142worker1 # [ 0.435463] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143lb # [ 0.389713] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1144client # [ 0.430034] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1145worker2 # [ 0.417925] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1146lb # [ 0.389743] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147worker1 # [ 0.435954] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1148client # [ 0.430084] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1149worker2 # [ 0.417936] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1150worker1 # [ 0.436144] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1151lb # [ 0.389760] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1152worker1 # [ 0.436160] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1153client # [ 0.430133] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1154worker2 # [ 0.417941] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1155lb # [ 0.390221] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1156worker1 # [ 0.436192] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1157lb # [ 0.390404] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1158client # [ 0.430182] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1159worker2 # [ 0.417986] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1160worker1 # [ 0.436797] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1161lb # [ 0.390420] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1162client # [ 0.430235] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1163worker2 # [ 0.418033] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1164lb # [ 0.390449] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1165worker1 # [ 0.436808] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1166client # [ 0.430283] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1167worker2 # [ 0.418081] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1168worker1 # [ 0.436813] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1169client # [ 0.430330] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1170worker2 # [ 0.418128] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1171worker1 # [ 0.436860] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1172lb # [ 0.409534] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1173client # [ 0.430393] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1174worker2 # [ 0.418174] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1175worker1 # [ 0.436908] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1176lb # [ 0.409724] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1177client # [ 0.430439] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1178worker2 # [ 0.418220] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1179lb # [ 0.409740] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1180worker1 # [ 0.436956] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1181client # [ 0.430461] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1182lb # [ 0.409769] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183worker2 # [ 0.418265] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1184worker1 # [ 0.437005] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1185client # [ 0.430483] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1186worker2 # [ 0.418310] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1187lb # [ 0.410236] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1188client # [ 0.430505] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1189worker1 # [ 0.437054] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1190lb # [ 0.410419] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1191worker2 # [ 0.418356] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1192client # [ 0.430527] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1193lb # [ 0.410435] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1194worker1 # [ 0.437103] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1195worker2 # [ 0.418428] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1196client # [ 0.430548] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1197lb # [ 0.410465] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1198worker1 # [ 0.437151] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1199worker2 # [ 0.418474] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1200client # [ 0.430570] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1201lb # [ 0.410483] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1202worker1 # [ 0.437197] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1203worker2 # [ 0.418495] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1204worker2 # [ 0.418517] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1205worker1 # [ 0.437244] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1206worker2 # [ 0.418538] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1207worker1 # [ 0.437329] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1208worker2 # [ 0.418559] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1209worker1 # [ 0.437384] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1210worker2 # [ 0.418580] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1211worker1 # [ 0.437407] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1212worker2 # [ 0.418601] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1213worker1 # [ 0.437428] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1214worker2 # [ 0.418625] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1215worker1 # [ 0.437449] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1216worker2 # [ 0.418647] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1217worker1 # [ 0.437471] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1218worker2 # [ 0.418669] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1219worker1 # [ 0.437493] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1220worker2 # [ 0.418691] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1221worker1 # [ 0.437514] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1222worker2 # [ 0.418714] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1223worker2 # [ 0.418735] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1224worker2 # [ 0.418756] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1225client # [ 0.466688] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1226worker2 # [ 0.418777] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1227client # [ 0.466716] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1228worker2 # [ 0.418797] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1229client # [ 0.466740] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1230worker2 # [ 0.418818] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1231worker2 # [ 0.418839] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1232client # [ 0.466764] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1233worker2 # [ 0.418864] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1234client # [ 0.466788] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1235lb # [ 0.423088] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1236worker2 # [ 0.418874] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1237client # [ 0.466811] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1238lb # [ 0.423299] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1239worker2 # [ 0.418879] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1240client # [ 0.466834] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1241lb # [ 0.423330] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1242worker2 # [ 0.419701] pci 0000:00:07.0: enabling device (0000 -> 0002)1243client # [ 0.466856] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1244lb # [ 0.423842] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1245client # [ 0.466879] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1246lb # [ 0.424034] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1247client # [ 0.466902] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1248lb # [ 0.424064] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1249client # [ 0.466926] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1250worker1 # [ 0.473650] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1251client # [ 0.466962] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1252lb # [ 0.424491] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1253worker1 # [ 0.473696] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1254client # [ 0.466972] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1255lb # [ 0.424674] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1256worker1 # [ 0.473721] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1257client # [ 0.466977] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1258lb # [ 0.424936] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1259client # [ 0.467824] pci 0000:00:07.0: enabling device (0000 -> 0002)1260worker1 # [ 0.473746] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1261lb # [ 0.425124] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1262worker1 # [ 0.473770] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1263lb # [ 0.425153] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1264worker1 # [ 0.473794] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1265lb # [ 0.425604] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1266worker1 # [ 0.473817] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1267lb # [ 0.425795] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1268worker1 # [ 0.473838] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1269lb # [ 0.425825] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1270worker1 # [ 0.473866] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1271worker2 # [ 0.479901] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 58792 usecs1272worker1 # [ 0.473888] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1273lb # [ 0.426285] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1274worker1 # [ 0.473911] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1275lb # [ 0.426472] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1276worker1 # [ 0.473943] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1277lb # [ 0.426502] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1278worker1 # [ 0.473952] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1279lb # [ 0.426999] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1280worker1 # [ 0.473957] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1281lb # [ 0.427306] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1282worker1 # [ 0.474822] pci 0000:00:07.0: enabling device (0000 -> 0002)1283lb # [ 0.427324] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1284lb # [ 0.427353] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1285lb # [ 0.427808] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1286lb # [ 0.428001] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1287lb # [ 0.428016] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1288lb # [ 0.428048] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1289lb # [ 0.428643] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1290lb # [ 0.428655] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1291lb # [ 0.428661] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1292lb # [ 0.428707] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1293lb # [ 0.428754] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1294lb # [ 0.428803] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1295lb # [ 0.428850] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1296lb # [ 0.428897] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1297lb # [ 0.428943] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1298lb # [ 0.428989] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1299client # [ 0.499870] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1300lb # [ 0.429036] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1301client # [ 0.502091] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1302lb # [ 0.429083] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1303lb # [ 0.429162] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1304lb # [ 0.429208] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1305worker2 # [ 0.500754] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1306worker1 # [ 0.502793] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1307lb # [ 0.429230] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1308worker2 # [ 0.502972] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1309worker1 # [ 0.505006] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1310lb # [ 0.429251] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1311lb # [ 0.429272] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1312lb # [ 0.429294] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1313lb # [ 0.429316] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1314lb # [ 0.429337] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1315lb # [ 0.429363] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1316lb # [ 0.429384] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1317lb # [ 0.429406] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1318client # [ 0.512487] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1319lb # [ 0.429428] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1320lb # [ 0.429450] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1321lb # [ 0.429472] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1322lb # [ 0.429493] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1323lb # [ 0.429514] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1324lb # [ 0.429534] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1325lb # [ 0.429555] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1326worker2 # [ 0.514226] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1327lb # [ 0.429576] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1328lb # [ 0.429602] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1329lb # [ 0.429612] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1330lb # [ 0.429616] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1331lb # [ 0.430441] pci 0000:00:07.0: enabling device (0000 -> 0002)1332client # [ 0.518710] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333client # [ 0.520851] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1334worker1 # [ 0.523163] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1335worker1 # [ 0.525322] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1336lb # [ 0.486984] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55221 usecs1337worker2 # [ 0.524695] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1338worker1 # [ 0.529107] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1339worker2 # [ 0.527056] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1340client # [ 0.530827] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1341worker2 # [ 0.530871] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1342client # [ 0.532703] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1343worker1 # [ 0.536638] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1344worker2 # [ 0.539085] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1345client # [ 0.542673] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1346client # [ 0.545343] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1347worker1 # [ 0.542719] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1348worker1 # [ 0.544824] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1349lb # [ 0.508292] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1350lb # [ 0.510481] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1351worker2 # [ 0.549156] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1352worker2 # [ 0.552648] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1353worker1 # [ 0.555116] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1354worker2 # [ 0.554579] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1355worker1 # [ 0.557092] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1356client # [ 0.555220] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1357client # [ 0.558506] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1358lb # [ 0.521921] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1359worker2 # [ 0.566222] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1360worker1 # [ 0.568947] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1361client # [ 0.571777] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1362client # [ 0.574597] msm_serial: driver initialized1363client # [ 0.574742] SuperH (H)SCI(F) driver initialized1364client # [ 0.574796] STM32 USART driver initialized1365lb # [ 0.536380] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1366lb # [ 0.538414] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1367worker1 # [ 0.581669] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1368worker2 # [ 0.581014] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1369lb # [ 0.542215] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1370worker1 # [ 0.584208] msm_serial: driver initialized1371worker1 # [ 0.584356] SuperH (H)SCI(F) driver initialized1372worker1 # [ 0.584408] STM32 USART driver initialized1373lb # [ 0.544958] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1374worker2 # [ 0.583782] msm_serial: driver initialized1375worker2 # [ 0.583915] SuperH (H)SCI(F) driver initialized1376worker2 # [ 0.583970] STM32 USART driver initialized1377lb # [ 0.555036] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1378lb # [ 0.557183] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1379client # [ 0.609168] loop: module loaded1380client # [ 0.609361] virtio_blk virtio2: 1/0/0 default/read/poll queues1381client # [ 0.610237] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1382lb # [ 0.567082] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1383lb # [ 0.570028] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1384client # [ 0.615243] megasas: 07.734.00.00-rc11385client # [ 0.615926] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1386worker2 # [ 0.616353] loop: module loaded1387worker1 # [ 0.618610] loop: module loaded1388worker2 # [ 0.616547] virtio_blk virtio2: 1/0/0 default/read/poll queues1389worker1 # [ 0.618809] virtio_blk virtio2: 1/0/0 default/read/poll queues1390worker2 # [ 0.617335] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1391worker1 # [ 0.619749] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1392lb # [ 0.580069] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1393lb # [ 0.582825] msm_serial: driver initialized1394client # [ 0.626623] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001395client # [ 0.626653] Intel/Sharp Extended Query Table at 0x00311396client # [ 0.628384] Using buffer write method1397lb # [ 0.587146] SuperH (H)SCI(F) driver initialized1398client # [ 0.628459] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1399lb # [ 0.587215] STM32 USART driver initialized1400worker1 # [ 0.630289] megasas: 07.734.00.00-rc11401worker2 # [ 0.624407] megasas: 07.734.00.00-rc11402worker1 # [ 0.631003] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1403worker2 # [ 0.625159] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1404worker1 # [ 0.633333] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001405worker2 # [ 0.627289] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406worker1 # [ 0.633360] Intel/Sharp Extended Query Table at 0x00311407worker2 # [ 0.627315] Intel/Sharp Extended Query Table at 0x00311408worker2 # [ 0.628943] Using buffer write method1409worker2 # [ 0.629006] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1410client # [ 0.639271] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001411worker2 # [ 0.630800] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412worker2 # [ 0.630823] Intel/Sharp Extended Query Table at 0x00311413client # [ 0.639300] Intel/Sharp Extended Query Table at 0x00311414client # [ 0.640714] Using buffer write method1415client # [ 0.640743] Concatenating MTD devices:1416client # [ 0.640747] (0): "0.flash"1417client # [ 0.640751] (1): "0.flash"1418client # [ 0.640764] into device "0.flash"1419worker1 # [ 0.642604] Using buffer write method1420worker1 # [ 0.642679] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1421worker2 # [ 0.648317] Using buffer write method1422worker2 # [ 0.648353] Concatenating MTD devices:1423worker2 # [ 0.648358] (0): "0.flash"1424worker2 # [ 0.648366] (1): "0.flash"1425worker2 # [ 0.648370] into device "0.flash"1426worker1 # [ 0.653586] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001427worker1 # [ 0.653634] Intel/Sharp Extended Query Table at 0x00311428worker1 # [ 0.655245] Using buffer write method1429worker1 # [ 0.655275] Concatenating MTD devices:1430worker1 # [ 0.655279] (0): "0.flash"1431worker1 # [ 0.655283] (1): "0.flash"1432worker1 # [ 0.655292] into device "0.flash"1433lb # [ 0.619125] loop: module loaded1434lb # [ 0.619328] virtio_blk virtio2: 1/0/0 default/read/poll queues1435lb # [ 0.620175] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1436lb # [ 0.627511] megasas: 07.734.00.00-rc11437lb # [ 0.628284] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1438lb # [ 0.630719] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001439lb # [ 0.630744] Intel/Sharp Extended Query Table at 0x00311440lb # [ 0.632379] Using buffer write method1441lb # [ 0.632444] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1442lb # [ 0.634242] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001443lb # [ 0.634265] Intel/Sharp Extended Query Table at 0x00311444lb # [ 0.652588] Using buffer write method1445lb # [ 0.652647] Concatenating MTD devices:1446lb # [ 0.652652] (0): "0.flash"1447lb # [ 0.652659] (1): "0.flash"1448lb # [ 0.652662] into device "0.flash"1449client # [ 0.920147] Freeing initrd memory: 26972K1450worker2 # [ 0.920036] Freeing initrd memory: 26944K1451client # [ 0.926292] tun: Universal TUN/TAP device driver, 1.61452worker2 # [ 0.926132] tun: Universal TUN/TAP device driver, 1.61453client # [ 0.930421] thunder_xcv, ver 1.01454client # [ 0.930473] thunder_bgx, ver 1.01455client # [ 0.930496] nicpf, ver 1.01456worker1 # [ 0.929675] Freeing initrd memory: 26976K1457worker2 # [ 0.930042] thunder_xcv, ver 1.01458worker2 # [ 0.930089] thunder_bgx, ver 1.01459client # [ 0.932281] e1000: Intel(R) PRO/1000 Network Driver1460worker2 # [ 0.930112] nicpf, ver 1.01461worker2 # [ 0.930672] e1000: Intel(R) PRO/1000 Network Driver1462client # [ 0.932292] e1000: Copyright (c) 1999-2006 Intel Corporation.1463worker2 # [ 0.930679] e1000: Copyright (c) 1999-2006 Intel Corporation.1464client # [ 0.932316] e1000e: Intel(R) PRO/1000 Network Driver1465worker1 # [ 0.935812] tun: Universal TUN/TAP device driver, 1.61466worker2 # [ 0.930703] e1000e: Intel(R) PRO/1000 Network Driver1467client # [ 0.932324] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1468worker2 # [ 0.930711] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1469client # [ 0.932353] igb: Intel(R) Gigabit Ethernet Network Driver1470worker2 # [ 0.930741] igb: Intel(R) Gigabit Ethernet Network Driver1471client # [ 0.932358] igb: Copyright (c) 2007-2014 Intel Corporation.1472worker2 # [ 0.930746] igb: Copyright (c) 2007-2014 Intel Corporation.1473client # [ 0.932381] igbvf: Intel(R) Gigabit Virtual Function Network Driver1474worker2 # [ 0.930788] igbvf: Intel(R) Gigabit Virtual Function Network Driver1475client # [ 0.932386] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1476client # [ 0.932520] sky2: driver version 1.301477worker2 # [ 0.930794] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1478worker2 # [ 0.930935] sky2: driver version 1.301479client # [ 0.934203] usbcore: registered new interface driver usb-storage1480client # [ 0.934300] usbcore: registered new interface driver usbserial_generic1481worker1 # [ 0.939543] thunder_xcv, ver 1.01482worker1 # [ 0.939611] thunder_bgx, ver 1.01483client # [ 0.934313] usbserial: USB Serial support registered for generic1484worker1 # [ 0.939631] nicpf, ver 1.01485worker1 # [ 0.940187] e1000: Intel(R) PRO/1000 Network Driver1486client # [ 0.935264] ehci-pci 0000:00:07.0: EHCI Host Controller1487worker2 # [ 0.940435] ehci-pci 0000:00:07.0: EHCI Host Controller1488worker1 # [ 0.940194] e1000: Copyright (c) 1999-2006 Intel Corporation.1489client # [ 0.935290] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11490worker2 # [ 0.940466] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11491worker1 # [ 0.940216] e1000e: Intel(R) PRO/1000 Network Driver1492client # [ 0.935496] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001493worker2 # [ 0.940703] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001494worker1 # [ 0.940226] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1495worker1 # [ 0.940254] igb: Intel(R) Gigabit Ethernet Network Driver1496worker2 # [ 0.943062] usbcore: registered new interface driver usb-storage1497client # [ 0.946689] hv_vmbus: registering driver hyperv_keyboard1498worker1 # [ 0.940260] igb: Copyright (c) 2007-2014 Intel Corporation.1499worker2 # [ 0.943145] usbcore: registered new interface driver usbserial_generic1500worker1 # [ 0.940282] igbvf: Intel(R) Gigabit Virtual Function Network Driver1501client # [ 0.948041] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001502worker2 # [ 0.943159] usbserial: USB Serial support registered for generic1503worker1 # [ 0.940288] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1504client # [ 0.949073] hub 1-0:1.0: USB hub found1505worker1 # [ 0.940417] sky2: driver version 1.301506client # [ 0.949554] hub 1-0:1.0: 6 ports detected1507worker2 # [ 0.946016] hv_vmbus: registering driver hyperv_keyboard1508worker2 # [ 0.947536] rtc-pl031 9010000.pl031: registered as rtc01509client # [ 0.950698] rtc-pl031 9010000.pl031: registered as rtc01510worker1 # [ 0.949119] usbcore: registered new interface driver usb-storage1511worker2 # [ 0.947563] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:37 UTC (1790140357)1512client # [ 0.950724] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:37 UTC (1790140357)1513worker1 # [ 0.949219] usbcore: registered new interface driver usbserial_generic1514client # [ 0.951044] i2c_dev: i2c /dev entries driver1515worker2 # [ 0.949533] i2c_dev: i2c /dev entries driver1516worker1 # [ 0.949232] usbserial: USB Serial support registered for generic1517worker1 # [ 0.950294] ehci-pci 0000:00:07.0: EHCI Host Controller1518worker1 # [ 0.950320] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11519worker1 # [ 0.950526] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001520worker1 # [ 0.955527] hv_vmbus: registering driver hyperv_keyboard1521client # [ 0.956379] sdhci: Secure Digital Host Controller Interface driver1522worker1 # [ 0.957123] rtc-pl031 9010000.pl031: registered as rtc01523worker2 # [ 0.953258] sdhci: Secure Digital Host Controller Interface driver1524client # [ 0.956391] sdhci: Copyright(c) Pierre Ossman1525worker2 # [ 0.953271] sdhci: Copyright(c) Pierre Ossman1526worker1 # [ 0.957151] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:37 UTC (1790140357)1527client # [ 0.956652] Synopsys Designware Multimedia Card Interface Driver1528worker1 # [ 0.957449] i2c_dev: i2c /dev entries driver1529worker2 # [ 0.953541] Synopsys Designware Multimedia Card Interface Driver1530client # [ 0.957037] sdhci-pltfm: SDHCI platform and OF driver helper1531worker2 # [ 0.953923] sdhci-pltfm: SDHCI platform and OF driver helper1532worker2 # [ 0.955452] hid: raw HID events driver (C) Jiri Kosina1533lb # [ 0.919500] Freeing initrd memory: 26948K1534worker2 # [ 0.955693] usbcore: registered new interface driver usbhid1535client # [ 0.961323] hid: raw HID events driver (C) Jiri Kosina1536worker2 # [ 0.955700] usbhid: USB HID core driver1537client # [ 0.961563] usbcore: registered new interface driver usbhid1538worker2 # [ 0.960246] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001539client # [ 0.961570] usbhid: USB HID core driver1540worker2 # [ 0.961284] hub 1-0:1.0: USB hub found1541worker2 # [ 0.961915] hub 1-0:1.0: 6 ports detected1542client # [ 0.964547] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1543worker1 # [ 0.962787] sdhci: Secure Digital Host Controller Interface driver1544worker1 # [ 0.962798] sdhci: Copyright(c) Pierre Ossman1545client # [ 0.966095] drop_monitor: Initializing network drop monitor service1546lb # [ 0.925658] tun: Universal TUN/TAP device driver, 1.61547worker1 # [ 0.963062] Synopsys Designware Multimedia Card Interface Driver1548client # [ 0.966285] NET: Registered PF_INET6 protocol family1549worker1 # [ 0.963427] sdhci-pltfm: SDHCI platform and OF driver helper1550client # [ 0.969379] Segment Routing with IPv61551worker1 # [ 0.964960] hid: raw HID events driver (C) Jiri Kosina1552worker2 # [ 0.963939] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1553client # [ 0.969398] In-situ OAM (IOAM) with IPv61554worker1 # [ 0.965195] usbcore: registered new interface driver usbhid1555worker2 # [ 0.965495] drop_monitor: Initializing network drop monitor service1556client # [ 0.969425] NET: Registered PF_PACKET protocol family1557worker1 # [ 0.965202] usbhid: USB HID core driver1558worker2 # [ 0.965655] NET: Registered PF_INET6 protocol family1559worker2 # [ 0.968759] Segment Routing with IPv61560worker1 # [ 0.970086] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001561worker2 # [ 0.968777] In-situ OAM (IOAM) with IPv61562worker1 # [ 0.970424] hub 1-0:1.0: USB hub found1563lb # [ 0.929528] thunder_xcv, ver 1.01564worker2 # [ 0.968809] NET: Registered PF_PACKET protocol family1565worker1 # [ 0.970504] hub 1-0:1.0: 6 ports detected1566lb # [ 0.929577] thunder_bgx, ver 1.01567lb # [ 0.929601] nicpf, ver 1.01568lb # [ 0.930176] e1000: Intel(R) PRO/1000 Network Driver1569client # [ 0.971293] 9pnet: Installing 9P2000 support1570lb # [ 0.930183] e1000: Copyright (c) 1999-2006 Intel Corporation.1571client # [ 0.971341] Key type dns_resolver registered1572worker2 # [ 0.970442] 9pnet: Installing 9P2000 support1573worker1 # [ 0.973336] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1574worker2 # [ 0.970494] Key type dns_resolver registered1575lb # [ 0.930208] e1000e: Intel(R) PRO/1000 Network Driver1576lb # [ 0.930216] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1577lb # [ 0.930247] igb: Intel(R) Gigabit Ethernet Network Driver1578worker1 # [ 0.975972] drop_monitor: Initializing network drop monitor service1579lb # [ 0.930253] igb: Copyright (c) 2007-2014 Intel Corporation.1580worker1 # [ 0.976160] NET: Registered PF_INET6 protocol family1581client # [ 0.977953] registered taskstats version 11582lb # [ 0.930292] igbvf: Intel(R) Gigabit Virtual Function Network Driver1583client # [ 0.978100] Loading compiled-in X.509 certificates1584lb # [ 0.930298] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1585worker1 # [ 0.978120] Segment Routing with IPv61586lb # [ 0.930435] sky2: driver version 1.301587worker1 # [ 0.978137] In-situ OAM (IOAM) with IPv61588worker1 # [ 0.978164] NET: Registered PF_PACKET protocol family1589worker1 # [ 0.979789] 9pnet: Installing 9P2000 support1590worker2 # [ 0.977236] registered taskstats version 11591lb # [ 0.939361] usbcore: registered new interface driver usb-storage1592worker1 # [ 0.979840] Key type dns_resolver registered1593worker2 # [ 0.977406] Loading compiled-in X.509 certificates1594lb # [ 0.939501] usbcore: registered new interface driver usbserial_generic1595lb # [ 0.939520] usbserial: USB Serial support registered for generic1596lb # [ 0.940162] hv_vmbus: registering driver hyperv_keyboard1597lb # [ 0.944488] ehci-pci 0000:00:07.0: EHCI Host Controller1598client # [ 0.986851] Demotion targets for Node 0: null1599client # [ 0.986963] Key type .fscrypt registered1600lb # [ 0.944518] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11601client # [ 0.986970] Key type fscrypt-provisioning registered1602lb # [ 0.944690] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001603client # [ 0.987075] ima: No TPM chip found, activating TPM-bypass!1604client # [ 0.987095] ima: Allocated hash algorithm: sha11605worker2 # [ 0.986349] Demotion targets for Node 0: null1606client # [ 0.987119] ima: No architecture policies found1607worker2 # [ 0.986466] Key type .fscrypt registered1608lb # [ 0.947272] rtc-pl031 9010000.pl031: registered as rtc01609worker1 # [ 0.986572] registered taskstats version 11610worker2 # [ 0.986474] Key type fscrypt-provisioning registered1611worker1 # [ 0.986753] Loading compiled-in X.509 certificates1612lb # [ 0.947303] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:37 UTC (1790140357)1613client # [ 0.991497] input: gpio-keys as /devices/platform/gpio-keys/input/input01614worker2 # [ 0.986575] ima: No TPM chip found, activating TPM-bypass!1615lb # [ 0.947646] i2c_dev: i2c /dev entries driver1616worker2 # [ 0.986594] ima: Allocated hash algorithm: sha11617worker2 # [ 0.986615] ima: No architecture policies found1618worker2 # [ 0.990755] input: gpio-keys as /devices/platform/gpio-keys/input/input01619worker1 # [ 0.995854] Demotion targets for Node 0: null1620lb # [ 0.952999] sdhci: Secure Digital Host Controller Interface driver1621worker1 # [ 0.995992] Key type .fscrypt registered1622lb # [ 0.953010] sdhci: Copyright(c) Pierre Ossman1623worker1 # [ 0.995999] Key type fscrypt-provisioning registered1624lb # [ 0.953284] Synopsys Designware Multimedia Card Interface Driver1625worker1 # [ 0.996101] ima: No TPM chip found, activating TPM-bypass!1626lb # [ 0.953661] sdhci-pltfm: SDHCI platform and OF driver helper1627worker1 # [ 0.996121] ima: Allocated hash algorithm: sha11628worker1 # [ 0.996144] ima: No architecture policies found1629lb # [ 0.954899] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001630lb # [ 0.955217] hub 1-0:1.0: USB hub found1631lb # [ 0.955239] hub 1-0:1.0: 6 ports detected1632worker1 # [ 1.000509] input: gpio-keys as /devices/platform/gpio-keys/input/input01633lb # [ 0.960511] hid: raw HID events driver (C) Jiri Kosina1634lb # [ 0.960776] usbcore: registered new interface driver usbhid1635lb # [ 0.960783] usbhid: USB HID core driver1636lb # [ 0.963777] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1637lb # [ 0.965391] drop_monitor: Initializing network drop monitor service1638lb # [ 0.965529] NET: Registered PF_INET6 protocol family1639client # [ 1.010065] clk: Disabling unused clocks1640lb # [ 0.968622] Segment Routing with IPv61641client # [ 1.010097] PM: genpd: Disabling unused power domains1642lb # [ 0.968656] In-situ OAM (IOAM) with IPv61643lb # [ 0.968683] NET: Registered PF_PACKET protocol family1644lb # [ 0.970333] 9pnet: Installing 9P2000 support1645lb # [ 0.970380] Key type dns_resolver registered1646worker2 # [ 1.008414] clk: Disabling unused clocks1647worker2 # [ 1.008448] PM: genpd: Disabling unused power domains1648client # [ 1.014269] Freeing unused kernel memory: 4736K1649client # [ 1.015225] Run /init as init process1650worker2 # [ 1.012702] Freeing unused kernel memory: 4736K1651worker2 # [ 1.012921] Run /init as init process1652lb # [ 0.977084] registered taskstats version 11653lb # [ 0.977228] Loading compiled-in X.509 certificates1654worker1 # [ 1.019740] clk: Disabling unused clocks1655worker1 # [ 1.019772] PM: genpd: Disabling unused power domains1656worker1 # [ 1.024066] Freeing unused kernel memory: 4736K1657worker1 # [ 1.024259] Run /init as init process1658lb # [ 0.986511] Demotion targets for Node 0: null1659client # [ 1.028981] systemd[1]: Successfully made /usr/ read-only.1660lb # [ 0.986662] Key type .fscrypt registered1661lb # [ 0.986669] Key type fscrypt-provisioning registered1662lb # [ 0.986776] ima: No TPM chip found, activating TPM-bypass!1663lb # [ 0.986796] ima: Allocated hash algorithm: sha11664lb # [ 0.986819] ima: No architecture policies found1665worker2 # [ 1.030074] systemd[1]: Successfully made /usr/ read-only.1666lb # [ 0.990940] input: gpio-keys as /devices/platform/gpio-keys/input/input01667worker1 # [ 1.039869] systemd[1]: Successfully made /usr/ read-only.1668lb # [ 1.008844] clk: Disabling unused clocks1669lb # [ 1.008873] PM: genpd: Disabling unused power domains1670lb # [ 1.012998] Freeing unused kernel memory: 4736K1671lb # [ 1.013205] Run /init as init process1672lb # [ 1.029223] systemd[1]: Successfully made /usr/ read-only.1673client # [ 1.194697] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker2 # [ 1.207788] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker1 # [ 1.217589] usb 1-1: new high-speed USB device number 2 using ehci-pci1676lb # [ 1.202972] usb 1-1: new high-speed USB device number 2 using ehci-pci1677client # [ 1.349096] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11678worker2 # [ 1.357966] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679worker1 # [ 1.367576] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11680worker2 # [ 1.364767] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1681worker2 # [ 1.371036] systemd[1]: Detected virtualization qemu.1682client # [ 1.363898] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1683worker2 # [ 1.372049] systemd[1]: Detected architecture arm64.1684worker2 # [ 1.372976] systemd[1]: Running in initrd.1685client # [ 1.376262] systemd[1]: Detected virtualization qemu.1686worker2 # [ 1.374316] systemd[1]: Initializing machine ID from random generator.1687worker2 # [ 1.375683] systemd[1]: Hostname set to <worker2>.1688client # [ 1.378442] systemd[1]: Detected architecture arm64.1689worker1 # [ 1.374541] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1690worker1 # [ 1.379230] systemd[1]: Detected virtualization qemu.1691client # [ 1.380474] systemd[1]: Running in initrd.1692worker1 # [ 1.380113] systemd[1]: Detected architecture arm64.1693worker1 # [ 1.380809] systemd[1]: Running in initrd.1694worker1 # [ 1.381905] systemd[1]: Initializing machine ID from random generator.1695worker1 # [ 1.382983] systemd[1]: Hostname set to <worker1>.1696client # [ 1.383127] systemd[1]: Initializing machine ID from random generator.1697client # [ 1.386010] systemd[1]: Hostname set to <client>.1698lb # [ 1.357582] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11699lb # [ 1.363786] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1700lb # [ 1.363848] systemd[1]: Detected virtualization qemu.1701lb # [ 1.363939] systemd[1]: Detected architecture arm64.1702lb # [ 1.363963] systemd[1]: Running in initrd.1703lb # [ 1.364980] systemd[1]: Initializing machine ID from random generator.1704lb # [ 1.365314] systemd[1]: Hostname set to <lb>.1705client # [ 1.430936] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker2 # [ 1.439908] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker1 # [ 1.453735] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708lb # [ 1.443147] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01709client # [ 1.558744] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker2 # [ 1.559810] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker1 # [ 1.573637] usb 1-2: new high-speed USB device number 3 using ehci-pci1712lb # [ 1.562957] usb 1-2: new high-speed USB device number 3 using ehci-pci1713worker1 # [ 1.626250] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714worker2 # [ 1.652467] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker1 # [ 1.688315] systemd[1]: Queued start job for default target Initrd Default Target.1716worker1 # [ 1.699770] systemd[1]: Created slice Slice /system/modprobe.1717worker1 # [ 1.701008] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1718worker1 # [ 1.702462] systemd[1]: Expecting device /dev/disk/by-label/nixos...1719worker1 # [ 1.703368] systemd[1]: Reached target Path Units.1720worker1 # [ 1.704058] systemd[1]: Reached target Slice Units.1721worker1 # [ 1.704831] systemd[1]: Reached target Swaps.1722worker1 # [ 1.705458] systemd[1]: Reached target Timer Units.1723worker1 # [ 1.706369] systemd[1]: Listening on D-Bus System Message Bus Socket.1724worker1 # [ 1.707430] systemd[1]: Listening on Journal Socket (/dev/log).1725worker1 # [ 1.708550] systemd[1]: Listening on Journal Sockets.1726worker1 # [ 1.709384] systemd[1]: Listening on udev Control Socket.1727worker1 # [ 1.710351] systemd[1]: Listening on udev Kernel Socket.1728worker1 # [ 1.711126] systemd[1]: Reached target Socket Units.1729worker1 # [ 1.713451] systemd[1]: Starting Create List of Static Device Nodes...1730worker1 # [ 1.714543] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1731worker2 # [ 1.713868] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21732worker2 # [ 1.716604] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01733worker1 # [ 1.725030] systemd[1]: Mounting Kernel Configuration File System...1734client # [ 1.721951] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21735client # [ 1.727988] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01736worker1 # [ 1.732034] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21737client # [ 1.733017] systemd[1]: bpf-restrict-fs: LSM BPF program attached1738lb # [ 1.691005] systemd[1]: bpf-restrict-fs: LSM BPF program attached1739worker1 # [ 1.739912] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01740worker2 # [ 1.740896] systemd[1]: Queued start job for default target Initrd Default Target.1741worker2 # [ 1.750357] systemd[1]: Created slice Slice /system/modprobe.1742worker1 # [ 1.753755] systemd[1]: Starting Journal Service...1743worker2 # [ 1.751651] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1744worker2 # [ 1.753225] systemd[1]: Expecting device /dev/disk/by-label/nixos...1745worker2 # [ 1.754497] systemd[1]: Reached target Path Units.1746worker2 # [ 1.755410] systemd[1]: Reached target Slice Units.1747worker2 # [ 1.756379] systemd[1]: Reached target Swaps.1748worker2 # [ 1.757183] systemd[1]: Reached target Timer Units.1749worker2 # [ 1.758319] systemd[1]: Listening on D-Bus System Message Bus Socket.1750worker1 # [ 1.761702] systemd[1]: Starting Load Kernel Modules...1751worker2 # [ 1.759657] systemd[1]: Listening on Journal Socket (/dev/log).1752worker1 # [ 1.762577] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1753worker2 # [ 1.761091] systemd[1]: Listening on Journal Sockets.1754worker2 # [ 1.762192] systemd[1]: Listening on udev Control Socket.1755worker2 # [ 1.763295] systemd[1]: Listening on udev Kernel Socket.1756worker2 # [ 1.764317] systemd[1]: Reached target Socket Units.1757lb # [ 1.723672] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21758worker2 # [ 1.767042] systemd[1]: Starting Create List of Static Device Nodes...1759lb # [ 1.728450] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01760worker2 # [ 1.768374] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1761worker1 # [ 1.774975] systemd[1]: Starting Coldplug All udev Devices...1762worker2 # [ 1.779906] systemd[1]: Mounting Kernel Configuration File System...1763worker2 # [ 1.784933] systemd[1]: Starting Journal Service...1764worker1 # [ 1.793693] systemd[1]: Finished Create List of Static Device Nodes.1765worker1 # [ 1.794748] systemd[1]: Mounted Kernel Configuration File System.1766worker2 # [ 1.803635] systemd[1]: Starting Load Kernel Modules...1767worker1 # [ 1.806051] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1768worker2 # [ 1.804669] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1769worker1 # [ 1.817087] systemd-journald[72]: Collecting audit messages is disabled.1770client # [ 1.824782] systemd[1]: Queued start job for default target Initrd Default Target.1771client # [ 1.832576] systemd[1]: Created slice Slice /system/modprobe.1772client # [ 1.833580] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1773client # [ 1.834631] systemd[1]: Expecting device /dev/disk/by-label/nixos...1774client # [ 1.835476] systemd[1]: Reached target Path Units.1775worker1 # [ 1.832562] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1776client # [ 1.836112] systemd[1]: Reached target Slice Units.1777client # [ 1.836752] systemd[1]: Reached target Swaps.1778client # [ 1.837334] systemd[1]: Reached target Timer Units.1779client # [ 1.838125] systemd[1]: Listening on D-Bus System Message Bus Socket.1780client # [ 1.839179] systemd[1]: Listening on Journal Socket (/dev/log).1781client # [ 1.840049] systemd[1]: Listening on Journal Sockets.1782client # [ 1.840807] systemd[1]: Listening on udev Control Socket.1783worker2 # [ 1.836320] systemd[1]: Starting Coldplug All udev Devices...1784client # [ 1.841625] systemd[1]: Listening on udev Kernel Socket.1785client # [ 1.842326] systemd[1]: Reached target Socket Units.1786worker1 # [ 1.841603] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1787client # [ 1.844498] systemd[1]: Starting Create List of Static Device Nodes...1788client # [ 1.845445] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1789lb # [ 1.806737] systemd[1]: Queued start job for default target Initrd Default Target.1790worker2 # [ 1.847897] systemd[1]: Finished Create List of Static Device Nodes.1791client # [ 1.854480] systemd[1]: Mounting Kernel Configuration File System...1792worker2 # [ 1.857407] systemd-journald[72]: Collecting audit messages is disabled.1793worker1 # [ 1.858605] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01794lb # [ 1.818700] systemd[1]: Created slice Slice /system/modprobe.1795worker1 # [ 1.858863] [drm] features: -virgl +edid -resource_blob -host_visible1796worker1 # [ 1.858874] [drm] features: -context_init1797lb # [ 1.819994] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1798client # [ 1.863011] systemd[1]: Starting Journal Service...1799worker1 # [ 1.859624] [drm] number of scanouts: 11800worker1 # [ 1.859643] [drm] number of cap sets: 01801lb # [ 1.821331] systemd[1]: Expecting device /dev/disk/by-label/nixos...1802lb # [ 1.822398] systemd[1]: Reached target Path Units.1803lb # [ 1.823269] systemd[1]: Reached target Slice Units.1804lb # [ 1.824102] systemd[1]: Reached target Swaps.1805lb # [ 1.824850] systemd[1]: Reached target Timer Units.1806worker2 # [ 1.863913] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1807lb # [ 1.825893] systemd[1]: Listening on D-Bus System Message Bus Socket.1808worker2 # [ 1.864419] systemd[1]: Mounted Kernel Configuration File System.1809worker1 # [ 1.866330] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1810worker1 # [ 1.869901] systemd[1]: Starting Create Static Device Nodes in /dev...1811lb # [ 1.827163] systemd[1]: Listening on Journal Socket (/dev/log).1812lb # [ 1.827311] systemd[1]: Listening on Journal Sockets.1813lb # [ 1.827451] systemd[1]: Listening on udev Control Socket.1814lb # [ 1.827566] systemd[1]: Listening on udev Kernel Socket.1815lb # [ 1.827590] systemd[1]: Reached target Socket Units.1816lb # [ 1.833637] systemd[1]: Starting Create List of Static Device Nodes...1817lb # [ 1.834804] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1818client # [ 1.878757] systemd[1]: Starting Load Kernel Modules...1819client # [ 1.878883] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1820worker2 # [ 1.882352] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1821worker1 # [ 1.881952] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1822worker1 # [ 1.881974] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01823lb # [ 1.846294] systemd[1]: Mounting Kernel Configuration File System...1824client # [ 1.892658] systemd[1]: Starting Coldplug All udev Devices...1825worker1 # [ 1.901850] Console: switching to colour frame buffer device 160x501826worker2 # [ 1.907861] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1827worker2 # [ 1.915384] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01828lb # [ 1.875183] systemd[1]: Starting Journal Service...1829worker2 # [ 1.915636] [drm] features: -virgl +edid -resource_blob -host_visible1830worker2 # [ 1.915645] [drm] features: -context_init1831client # [ 1.918834] systemd[1]: Finished Create List of Static Device Nodes.1832client # [ 1.919877] systemd[1]: Mounted Kernel Configuration File System.1833client # [ 1.923085] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1834lb # [ 1.883061] systemd[1]: Starting Load Kernel Modules...1835lb # [ 1.884040] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1836worker2 # [ 1.928425] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1837lb # [ 1.891126] systemd[1]: Starting Coldplug All udev Devices...1838worker2 # [ 1.931190] systemd[1]: Starting Create Static Device Nodes in /dev...1839worker1 # [ 1.934542] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1840worker1 # [ 1.936298] systemd[1]: Finished Create Static Device Nodes in /dev.1841worker1 # [ 1.936571] systemd[1]: Reached target Preparation for Local File Systems.1842worker1 # [ 1.936619] systemd[1]: Reached target Local File Systems.1843worker1 # [ 1.940760] systemd[1]: Starting Rule-based Manager for Device Events and Files...1844client # [ 1.941706] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1845worker2 # [ 1.940506] [drm] number of scanouts: 11846worker2 # [ 1.940538] [drm] number of cap sets: 01847client # [ 1.949013] systemd-journald[72]: Collecting audit messages is disabled.1848worker2 # [ 1.944151] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1849worker2 # [ 1.944167] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01850lb # [ 1.911049] systemd[1]: Finished Create List of Static Device Nodes.1851lb # [ 1.912051] systemd[1]: Mounted Kernel Configuration File System.1852worker1 # [ 1.953876] systemd[1]: Finished Load Kernel Modules.1853client # [ 1.958656] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1854lb # [ 1.919547] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1855worker1 # [ 1.961945] systemd[1]: Starting Apply Kernel Variables...1856client # [ 1.975961] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01857client # [ 1.976212] [drm] features: -virgl +edid -resource_blob -host_visible1858client # [ 1.976223] [drm] features: -context_init1859client # [ 1.977014] [drm] number of scanouts: 11860worker2 # [ 1.968039] Console: switching to colour frame buffer device 160x501861client # [ 1.977034] [drm] number of cap sets: 01862worker2 # [ 1.974863] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1863worker1 # [ 1.999373] systemd[1]: Started Journal Service.1864client # [ 1.998984] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1865lb # [ 1.960480] systemd-journald[72]: Collecting audit messages is disabled.1866client # [ 1.999010] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01867worker1 # [ 1.990675] systemd-modules-load[73]: Inserted module 'dm_mod'1868client # [ 2.005201] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1869worker1 # [ 1.991967] systemd-modules-load[73]: Module 'virtio_balloon' is built in1870worker2 # [ 2.000948] systemd[1]: Finished Create Static Device Nodes in /dev.1871worker2 # [ 2.001195] systemd[1]: Reached target Preparation for Local File Systems.1872worker2 # [ 2.001231] systemd[1]: Reached target Local File Systems.1873worker2 # [ 2.008194] systemd[1]: Starting Rule-based Manager for Device Events and Files...1874client # [ 2.010985] systemd[1]: Starting Create Static Device Nodes in /dev...1875worker1 # [ 2.004440] systemd-modules-load[73]: Module 'virtio_console' is built in1876worker1 # [ 2.005595] systemd-modules-load[73]: Inserted module 'virtio_gpu'1877worker2 # [ 2.015839] systemd[1]: Finished Load Kernel Modules.1878worker1 # [ 2.006619] systemd-modules-load[73]: Module 'virtio_rng' is built in1879worker1 # [ 2.007607] systemd[1]: Starting Create System Files and Directories...1880lb # [ 1.983685] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1881lb # [ 1.985414] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1882worker2 # [ 2.014734] systemd-modules-load[73]: Inserted module 'dm_mod'1883client # [ 2.026881] Console: switching to colour frame buffer device 160x501884worker2 # [ 2.020189] systemd-modules-load[73]: Module 'virtio_balloon' is built in1885lb # [ 1.991608] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1886worker2 # [ 2.021324] systemd-modules-load[73]: Module 'virtio_console' is built in1887worker2 # [ 2.022449] systemd-modules-load[73]: Inserted module 'virtio_gpu'1888worker2 # [ 2.023416] systemd-modules-load[73]: Module 'virtio_rng' is built in1889worker1 # [ 2.024154] systemd[1]: Finished Apply Kernel Variables.1890lb # [ 1.999334] systemd[1]: Starting Create Static Device Nodes in /dev...1891worker2 # [ 2.044414] systemd[1]: Starting Apply Kernel Variables...1892worker2 # [ 2.044619] systemd[1]: Started Journal Service.1893lb # [ 2.011716] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01894lb # [ 2.011962] [drm] features: -virgl +edid -resource_blob -host_visible1895lb # [ 2.011973] [drm] features: -context_init1896client # [ 2.056191] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1897lb # [ 2.012681] [drm] number of scanouts: 11898lb # [ 2.012700] [drm] number of cap sets: 01899lb # [ 2.019285] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1900worker1 # [ 2.050915] systemd-udevd[78]: Using default interface naming scheme 'v261'.1901lb # [ 2.019302] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01902client # [ 2.063980] systemd[1]: Finished Load Kernel Modules.1903worker2 # [ 2.054010] systemd[1]: Starting Create System Files and Directories...1904client # [ 2.075123] systemd[1]: Starting Apply Kernel Variables...1905worker1 # [ 2.070082] systemd[1]: Finished Create System Files and Directories.1906client # [ 2.090713] systemd[1]: Finished Create Static Device Nodes in /dev.1907client # [ 2.090957] systemd[1]: Reached target Preparation for Local File Systems.1908client # [ 2.090994] systemd[1]: Reached target Local File Systems.1909client # [ 2.096332] systemd[1]: Starting Rule-based Manager for Device Events and Files...1910worker2 # [ 2.088914] systemd[1]: Finished Apply Kernel Variables.1911worker1 # [ 2.088918] systemd[1]: Started Rule-based Manager for Device Events and Files.1912lb # [ 2.054276] Console: switching to colour frame buffer device 160x501913lb # [ 2.061060] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1914lb # [ 2.067026] systemd[1]: Finished Create Static Device Nodes in /dev.1915lb # [ 2.067268] systemd[1]: Reached target Preparation for Local File Systems.1916lb # [ 2.067300] systemd[1]: Reached target Local File Systems.1917lb # [ 2.075360] systemd[1]: Starting Rule-based Manager for Device Events and Files...1918worker2 # [ 2.105653] systemd[1]: Finished Create System Files and Directories.1919client # [ 2.125649] systemd[1]: Started Journal Service.1920worker2 # [ 2.113373] systemd-udevd[78]: Using default interface naming scheme 'v261'.1921client # [ 2.115697] systemd-modules-load[73]: Inserted module 'dm_mod'1922client # [ 2.121485] systemd-modules-load[73]: Module 'virtio_balloon' is built in1923client # [ 2.132320] systemd-modules-load[73]: Module 'virtio_console' is built in1924lb # [ 2.103163] systemd[1]: Finished Load Kernel Modules.1925lb # [ 2.107283] systemd[1]: Starting Apply Kernel Variables...1926client # [ 2.136304] systemd-modules-load[73]: Inserted module 'virtio_gpu'1927worker2 # [ 2.140730] systemd[1]: Started Rule-based Manager for Device Events and Files.1928client # [ 2.144370] systemd-modules-load[73]: Module 'virtio_rng' is built in1929client # [ 2.146704] systemd[1]: Starting Create System Files and Directories...1930client # [ 2.154035] systemd[1]: Finished Apply Kernel Variables.1931worker1 # [ 2.168130] systemd[1]: Starting Virtual Console Setup...1932client # [ 2.173672] systemd[1]: Finished Create System Files and Directories.1933lb # [ 2.151079] systemd[1]: Started Journal Service.1934lb # [ 2.139451] systemd-modules-load[73]: Inserted module 'dm_mod'1935client # [ 2.185462] systemd-udevd[80]: Using default interface naming scheme 'v261'.1936lb # [ 2.144258] systemd-modules-load[73]: Module 'virtio_balloon' is built in1937lb # [ 2.145412] systemd-modules-load[73]: Module 'virtio_console' is built in1938lb # [ 2.146482] systemd-modules-load[73]: Inserted module 'virtio_gpu'1939lb # [ 2.147488] systemd-modules-load[73]: Module 'virtio_rng' is built in1940worker2 # [ 2.199884] systemd[1]: Starting Virtual Console Setup...1941lb # [ 2.164515] systemd[1]: Starting Create System Files and Directories...1942client # [ 2.217105] systemd[1]: Started Rule-based Manager for Device Events and Files.1943lb # [ 2.175755] systemd-udevd[78]: Using default interface naming scheme 'v261'.1944lb # [ 2.181350] systemd[1]: Finished Apply Kernel Variables.1945lb # [ 2.193304] systemd[1]: Finished Create System Files and Directories.1946worker1 # [ 2.238285] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1947worker1 # [ 2.241994] systemd[1]: Finished Virtual Console Setup.1948worker2 # [ 2.252475] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1949worker2 # [ 2.256163] systemd[1]: Finished Virtual Console Setup.1950lb # [ 2.217988] systemd[1]: Started Rule-based Manager for Device Events and Files.1951client # [ 2.280467] systemd[1]: Starting Virtual Console Setup...1952lb # [ 2.273939] systemd[1]: Starting Virtual Console Setup...1953client # [ 2.328557] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954client # [ 2.332182] systemd[1]: Finished Virtual Console Setup.1955lb # [ 2.324514] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1956lb # [ 2.327940] systemd[1]: Finished Virtual Console Setup.1957worker2 # [ 2.837316] systemd[1]: Finished Coldplug All udev Devices.1958worker2 # [ 2.838292] systemd[1]: Reached target System Initialization.1959worker2 # [ 2.839123] systemd[1]: Reached target Basic System.1960worker1 # [ 2.853164] systemd[1]: Finished Coldplug All udev Devices.1961worker1 # [ 2.854758] systemd[1]: Reached target System Initialization.1962worker1 # [ 2.855613] systemd[1]: Reached target Basic System.1963client # [ 2.932784] systemd[1]: Finished Coldplug All udev Devices.1964client # [ 2.933767] systemd[1]: Reached target System Initialization.1965client # [ 2.934600] systemd[1]: Reached target Basic System.1966worker1 # [ 2.976258] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1967lb # [ 2.939443] systemd[1]: Finished Coldplug All udev Devices.1968lb # [ 2.940466] systemd[1]: Reached target System Initialization.1969lb # [ 2.941261] systemd[1]: Reached target Basic System.1970worker2 # [ 3.019699] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 3.031643] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972worker1 # [ 3.040205] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line.1973worker2 # [ 3.045637] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1974worker2 # [ 3.052236] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1975client # [ 3.080547] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976worker1 # [ 3.100424] systemd[1]: Found device /dev/disk/by-label/nixos.1977client # [ 3.107308] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1978worker1 # [ 3.108321] systemd[1]: Reached target Initrd Root Device.1979worker1 # [ 3.114049] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1980client # [ 3.118395] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1981lb # [ 3.076414] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1982lb # [ 3.081015] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1983lb # [ 3.091377] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1984worker2 # [ 3.142092] systemd[1]: Found device /dev/disk/by-label/nixos.1985worker2 # [ 3.147892] systemd[1]: Reached target Initrd Root Device.1986worker2 # [ 3.151883] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1987worker1 # [ 3.167437] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1988worker1 # [ 3.175316] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1989worker1 # [ 3.180174] systemd[1]: Mounting /sysroot...1990worker2 # [ 3.205358] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1991client # [ 3.206749] systemd[1]: Found device /dev/disk/by-label/nixos.1992client # [ 3.208732] systemd[1]: Reached target Initrd Root Device.1993client # [ 3.211332] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1994worker2 # [ 3.212210] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995worker2 # [ 3.215826] systemd[1]: Mounting /sysroot...1996worker1 # [ 3.229233] EXT4-fs (vda): mounted filesystem f896596c-c5ce-4568-8087-8dcfd162026c r/w with ordered data mode. Quota mode: none.1997worker1 # [ 3.220962] systemd[1]: Mounted /sysroot.1998worker1 # [ 3.222849] systemd[1]: Reached target Initrd Root File System.1999worker1 # [ 3.224200] systemd[1]: Starting Mountpoints Configured in the Real Root...2000lb # [ 3.210536] systemd[1]: Found device /dev/disk/by-label/nixos.
loading 2,398 hidden lines…
4399worker2 # [ 52.586611] nix-grpc-daemon[873]: time=2026-09-23T05:13:29.150Z level=INFO msg="Uploading 1 narinfos"4400lb # [ 52.552785] niks3-server[1013]: 2026/09/23 05:13:29 INFO Signed narinfos id=8 count=14401lb # [ 52.556002] niks3-server[1013]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4402lb # [ 52.575828] niks3-server[1013]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4403worker2 # [ 52.626676] nix-grpc-daemon[873]: time=2026-09-23T05:13:29.190Z level=INFO msg="Upload complete. (260ms)"4404lb # [ 52.587755] niks3-server[1013]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4405lb # [ 52.593219] niks3-server[1013]: 2026/09/23 05:13:29 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4406lb # [ 52.598012] niks3-server[1013]: 2026/09/23 05:13:29 INFO Completed upload id=84407lb # [ 52.601134] niks3-server[1013]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4408worker2 # [ 52.642123] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14409worker2 # [ 52.659747] nix-daemon[858]: reaped child process 1010, status = succeeded4410client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.20 seconds)4411client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4412client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.16 seconds)4413lb: must succeed: curl -sf localhost:9901/clusters4414lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4415lb: must succeed: curl -sf localhost:9901/clusters4416lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4417worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4418worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4419(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.72 seconds)4420subtest: outputs read back through any worker carry the cache signature4421client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24422worker2 # [ 53.251828] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:454804423client # don't know how to build these paths:4424client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4425worker2 # [ 53.261872] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454804426worker2 # [ 53.286191] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04427worker2 # [ 53.292623] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=14428worker1 # [ 53.298189] nix-grpc-daemon[768]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404429worker1 # [ 53.315492] nix-grpc-daemon[768]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376404430client # copying 1 paths to 'lb:50051'...4431client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4432worker1 # [ 53.332211] nix-grpc-daemon[768]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376404433worker1 # [ 53.348134] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4434worker1 # [ 53.378612] nix-grpc-daemon[768]: ts=2026-09-23T05:13:29Z level=info event=niks3_push_started pid=8624435worker1 # [ 53.555512] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.111Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4436worker1 # [ 53.673192] nix-daemon[835]: accepted connection from pid 875, user nix-grpc-daemon (trusted)4437worker1 # [ 53.693312] nix-daemon[835]: reaped child process 878, status = succeeded4438lb # [ 53.657312] niks3-server[1013]: 2026/09/23 05:13:30 INFO Received uploads request method=POST path=/api/pending_closures4439worker1 # [ 53.711631] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.266Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4440worker1 # [ 53.715620] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.266Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4441worker1 # [ 53.753199] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.308Z level=INFO msg="Uploading 1 narinfos"4442lb # [ 53.700140] niks3-server[1013]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4443lb # [ 53.713723] niks3-server[1013]: 2026/09/23 05:13:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4444lb # [ 53.726377] niks3-server[1013]: 2026/09/23 05:13:30 INFO Signed narinfos id=9 count=14445lb # [ 53.736368] niks3-server[1013]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4446lb # [ 53.739728] niks3-server[1013]: 2026/09/23 05:13:30 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4447worker1 # [ 53.789105] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.345Z level=INFO msg="Upload complete. (229ms)"4448lb # [ 53.747152] niks3-server[1013]: 2026/09/23 05:13:30 INFO Completed upload id=94449lb # [ 53.750080] niks3-server[1013]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4450worker1 # [ 53.797500] nix-grpc-daemon[768]: ts=2026-09-23T05:13:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=5124451worker1 # [ 53.811582] nix-grpc-daemon[768]: ts=2026-09-23T05:13:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404452worker1 # [ 53.818215] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4453worker1 # [ 53.823046] nix-daemon[835]: reaped child process 860, status = succeeded4454client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4455worker1 # [ 53.834246] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4456client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4457client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4458client # farm-a> LOG-farm-a-sig4459worker1 # [ 55.471359] nix-daemon[835]: reaped child process 882, status = succeeded4460worker1 # [ 55.570727] nix-daemon[835]: accepted connection from pid 898, user nix-grpc-daemon (trusted)4461worker1 # [ 55.591501] nix-daemon[835]: reaped child process 901, status = succeeded4462lb # [ 55.551478] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received uploads request method=POST path=/api/pending_closures4463worker1 # [ 55.604391] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4464worker1 # [ 55.605859] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.160Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4465lb # [ 55.602295] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4466worker1 # [ 55.655020] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.210Z level=INFO msg="Uploading 1 narinfos"4467lb # [ 55.610352] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4468lb # [ 55.623415] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4469lb # [ 55.631719] niks3-server[1013]: 2026/09/23 05:13:32 INFO Signed narinfos id=10 count=14470lb # [ 55.640545] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4471worker1 # [ 55.688238] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.244Z level=INFO msg="Upload complete. (241ms)"4472lb # [ 55.646416] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4473lb # [ 55.650556] niks3-server[1013]: 2026/09/23 05:13:32 INFO Completed upload id=104474lb # [ 55.658643] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4475worker1 # [ 55.700941] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14476worker2 # [ 55.709421] nix-grpc-daemon[771]: ts=2026-09-23T05:13:32Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04477worker1 # [ 55.718298] nix-daemon[835]: reaped child process 880, status = succeeded4478worker2 # [ 55.715649] nix-grpc-daemon[771]: ts=2026-09-23T05:13:32Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14479worker1 # [ 55.755468] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404480worker1 # [ 55.771680] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376404481worker1 # [ 55.788079] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.344Z level=INFO msg="All 1 paths already cached"4482client # copying 1 paths to 'lb:50051'...4483client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4484worker1 # [ 55.799216] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376404485worker1 # [ 55.814430] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4486worker1 # [ 55.965660] nix-daemon[835]: accepted connection from pid 905, user nix-grpc-daemon (trusted)4487worker1 # [ 55.984953] nix-daemon[835]: reaped child process 908, status = succeeded4488lb # [ 55.948663] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received uploads request method=POST path=/api/pending_closures4489worker1 # [ 56.000271] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.556Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4490worker1 # [ 56.002123] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.556Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4491lb # [ 55.978084] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4492worker1 # [ 56.027694] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.583Z level=INFO msg="Uploading 1 narinfos"4493lb # [ 55.991302] niks3-server[1013]: 2026/09/23 05:13:32 INFO Signed narinfos id=11 count=14494lb # [ 56.001952] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4495lb # [ 56.014943] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4496worker1 # [ 56.065747] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.621Z level=INFO msg="Upload complete. (235ms)"4497lb # [ 56.023884] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4498lb # [ 56.029272] niks3-server[1013]: 2026/09/23 05:13:32 INFO Completed upload id=114499worker1 # [ 56.072934] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=6244500lb # [ 56.035358] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4501worker1 # [ 56.084296] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404502worker1 # [ 56.097129] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4503worker1 # [ 56.102576] nix-daemon[835]: reaped child process 903, status = succeeded4504client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4505worker1 # [ 56.113658] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4506client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4507client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4508client # farm-b> LOG-farm-b-sig4509worker1 # [ 57.572866] nix-daemon[835]: reaped child process 912, status = succeeded4510worker1 # [ 57.696456] nix-daemon[835]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4511worker1 # [ 57.718006] nix-daemon[835]: reaped child process 931, status = succeeded4512lb # [ 57.675394] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received uploads request method=POST path=/api/pending_closures4513worker1 # [ 57.728520] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.284Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4514worker1 # [ 57.730118] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.284Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4515lb # [ 57.720290] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4516worker1 # [ 57.774495] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.330Z level=INFO msg="Uploading 1 narinfos"4517lb # [ 57.734520] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4518lb # [ 57.744349] niks3-server[1013]: 2026/09/23 05:13:34 INFO Signed narinfos id=12 count=14519lb # [ 57.752433] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4520lb # [ 57.763181] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4521lb # [ 57.775761] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4522worker1 # [ 57.816912] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.373Z level=INFO msg="Upload complete. (287ms)"4523lb # [ 57.783813] niks3-server[1013]: 2026/09/23 05:13:34 INFO Completed upload id=124524lb # [ 57.787424] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4525worker1 # [ 57.829436] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14526worker2 # [ 57.838137] nix-grpc-daemon[771]: ts=2026-09-23T05:13:34Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04527worker2 # [ 57.844730] nix-grpc-daemon[771]: ts=2026-09-23T05:13:34Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14528worker1 # [ 57.851499] nix-daemon[835]: reaped child process 910, status = succeeded4529worker1 # [ 57.883123] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404530worker1 # [ 57.900762] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376404531worker1 # [ 57.916634] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.472Z level=INFO msg="All 2 paths already cached"4532client # copying 1 paths to 'lb:50051'...4533client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4534worker1 # [ 57.928531] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376404535worker1 # [ 57.942279] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4536worker1 # [ 58.100414] nix-daemon[835]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4537worker1 # [ 58.120886] nix-daemon[835]: reaped child process 938, status = succeeded4538lb # [ 58.083244] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received uploads request method=POST path=/api/pending_closures4539worker1 # [ 58.136090] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.692Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4540worker1 # [ 58.137763] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.692Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4541worker1 # [ 58.172773] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.728Z level=INFO msg="Uploading 1 narinfos"4542lb # [ 58.123290] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4543lb # [ 58.133989] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4544lb # [ 58.141860] niks3-server[1013]: 2026/09/23 05:13:34 INFO Signed narinfos id=13 count=14545lb # [ 58.147142] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4546worker1 # [ 58.202352] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.758Z level=INFO msg="Upload complete. (242ms)"4547lb # [ 58.160569] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4548lb # [ 58.170505] niks3-server[1013]: 2026/09/23 05:13:34 INFO Completed upload id=134549lb # [ 58.172770] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4550worker1 # [ 58.213089] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=7524551worker1 # [ 58.228341] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404552worker1 # [ 58.234486] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4553worker1 # [ 58.239458] nix-daemon[835]: reaped child process 933, status = succeeded4554client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4555worker1 # [ 58.251014] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4556client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4557lb # [ 58.364822] envoy[900]: conn peer=[2001:db8:1::1]:58594 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9519 tx=4403 ms=55134558lb # [ 58.370078] envoy[900]: conn peer=[2001:db8:1::1]:58596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=634559lb # [ 58.374841] envoy[900]: conn peer=[2001:db8:1::1]:58598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1896 ms=19574560client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4561client # farm-top> LOG-farm-top-sig4562worker1 # [ 59.767047] nix-daemon[835]: reaped child process 942, status = succeeded4563worker1 # [ 59.837426] nix-daemon[835]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4564worker1 # [ 59.858760] nix-daemon[835]: reaped child process 961, status = succeeded4565lb # [ 59.815666] niks3-server[1013]: 2026/09/23 05:13:36 INFO Received uploads request method=POST path=/api/pending_closures4566worker1 # [ 59.868497] nix-grpc-daemon[862]: time=2026-09-23T05:13:36.424Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4567worker1 # [ 59.870026] nix-grpc-daemon[862]: time=2026-09-23T05:13:36.424Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4568lb # [ 59.854112] niks3-server[1013]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4569worker1 # [ 59.914486] nix-grpc-daemon[862]: time=2026-09-23T05:13:36.470Z level=INFO msg="Uploading 1 narinfos"4570lb # [ 59.872672] niks3-server[1013]: 2026/09/23 05:13:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4571lb # [ 59.881521] niks3-server[1013]: 2026/09/23 05:13:36 INFO Signed narinfos id=14 count=14572lb # [ 59.888770] niks3-server[1013]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4573lb # [ 59.896627] niks3-server[1013]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4574lb # [ 59.906489] niks3-server[1013]: 2026/09/23 05:13:36 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4575lb # [ 59.910570] niks3-server[1013]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4576lb # [ 59.915605] niks3-server[1013]: 2026/09/23 05:13:36 INFO Completed upload id=144577worker1 # [ 59.957655] nix-grpc-daemon[862]: time=2026-09-23T05:13:36.513Z level=INFO msg="Upload complete. (290ms)"4578worker1 # [ 59.969198] nix-grpc-daemon[768]: ts=2026-09-23T05:13:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14579worker1 # [ 59.989834] nix-daemon[835]: reaped child process 940, status = succeeded4580client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.99 seconds)4581client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4582client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4583client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4584worker1 # [ 60.343089] nix-grpc-daemon[768]: ts=2026-09-23T05:13:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:367664585worker1 # [ 60.386596] nix-grpc-daemon[768]: ts=2026-09-23T05:13:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:367664586client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds)4587client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4588worker2 # [ 60.638105] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:557084589client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4590client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4591client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4592client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4593worker2 # [ 60.685612] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:557084594worker2 # [ 60.695296] nix-grpc-daemon[771]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4595worker2 # [ 60.713623] nix-grpc-daemon[771]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4596worker2 # [ 60.730297] nix-grpc-daemon[771]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4597client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.32 seconds)4598client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4599client # finding garbage collector roots...4600client # removing stale temporary roots file "/nix/var/nix/temproots/906"4601client # removing stale temporary roots file "/nix/var/nix/temproots/898"4602client # removing stale temporary roots file "/nix/var/nix/temproots/849"4603client # removing stale temporary roots file "/nix/var/nix/temproots/926"4604client # removing stale temporary roots file "/nix/var/nix/temproots/918"4605client # removing stale temporary roots file "/nix/var/nix/temproots/886"4606client # removing stale temporary roots file "/nix/var/nix/temproots/941"4607client # removing stale temporary roots file "/nix/var/nix/temproots/865"4608client # deleting specified paths...4609client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4610client # deleting unused links...4611client # note: hard linking is currently saving 0.0 KiB4612worker1 # [ 61.262294] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376404613worker1 # [ 61.271922] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376404614worker1 # [ 61.285778] nix-grpc-daemon[862]: time=2026-09-23T05:13:37.841Z level=INFO msg="All 1 paths already cached"4615worker1 # [ 61.294935] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376404616worker2 # [ 61.304546] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:454804617client # copying 3 paths...4618worker1 # [ 61.322557] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376404619client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4620worker2 # [ 61.358569] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:454804621worker2 # [ 61.367410] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=2324622worker2 # [ 61.380583] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:454804623worker2 # [ 61.388333] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=1764624worker1 # [ 61.398466] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376404625worker1 # [ 61.406082] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_out=1284626client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4627client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4628client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.70 seconds)4629(finished: subtest: outputs read back through any worker carry the cache signature, in 8.45 seconds)4630subtest: two clients wanting the same drv build it once4631client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4632client # [ 61.528728] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4633client # Running as unit: dedup1.service; invocation ID: 8453cebab28a420cb6a4171e9dbba7824634client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4635client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4636client # [ 61.722915] nix[994]: don't know how to build these paths:4637client # [ 61.725043] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4638worker2 # [ 61.721634] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:454804639worker2 # [ 61.737679] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04640worker2 # [ 61.747179] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14641worker2 # [ 61.782993] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804642worker2 # [ 61.802388] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454804643client # [ 61.820207] nix[994]: copying 1 paths to 'lb:50051'...4644worker2 # [ 61.819323] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454804645client # [ 61.822824] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4646worker2 # [ 61.837397] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4647worker2 # [ 62.002063] nix-daemon[858]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4648worker2 # [ 62.025628] nix-daemon[858]: reaped child process 1054, status = succeeded4649lb # [ 61.991140] niks3-server[1013]: 2026/09/23 05:13:38 INFO Received uploads request method=POST path=/api/pending_closures4650worker2 # [ 62.080494] nix-grpc-daemon[873]: time=2026-09-23T05:13:38.642Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4651worker2 # [ 62.083711] nix-grpc-daemon[873]: time=2026-09-23T05:13:38.642Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4652worker2 # [ 62.125092] nix-grpc-daemon[873]: time=2026-09-23T05:13:38.688Z level=INFO msg="Uploading 1 narinfos"4653lb # [ 62.071727] niks3-server[1013]: 2026/09/23 05:13:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4654lb # [ 62.087581] niks3-server[1013]: 2026/09/23 05:13:38 INFO Signed narinfos id=15 count=14655lb # [ 62.096525] niks3-server[1013]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4656lb # [ 62.106700] niks3-server[1013]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4657lb # [ 62.117329] niks3-server[1013]: 2026/09/23 05:13:38 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4658lb # [ 62.127305] niks3-server[1013]: 2026/09/23 05:13:38 INFO Completed upload id=154659worker2 # [ 62.169721] nix-grpc-daemon[873]: time=2026-09-23T05:13:38.733Z level=INFO msg="Upload complete. (313ms)"4660lb # [ 62.130789] niks3-server[1013]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4661worker2 # [ 62.180463] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=4324662worker2 # [ 62.200783] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804663worker2 # [ 62.211462] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4664worker2 # [ 62.218674] nix-daemon[858]: reaped child process 1049, status = succeeded4665client # [ 62.231334] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4666worker2 # [ 62.234628] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4667client # [ 62.629405] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4668client # Running as unit: dedup2.service; invocation ID: c3302b0c7757426eb33bd07889e5d4034669client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4670worker1 # [ 62.858300] nix-grpc-daemon[768]: ts=2026-09-23T05:13:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376404671client # [ 62.868941] nix[1010]: don't know how to build these paths:4672client # [ 62.869949] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4673worker2 # [ 62.885238] nix-grpc-daemon[771]: ts=2026-09-23T05:13:39Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04674worker2 # [ 62.892703] nix-grpc-daemon[771]: ts=2026-09-23T05:13:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804675client # [ 62.900993] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4676client: must succeed: sleep 24677client: (finished: must succeed: sleep 2, in 2.06 seconds)4678client: waiting for success: ! systemctl is-active dedup1 dedup24679worker2 # [ 66.268348] nix-daemon[858]: reaped child process 1058, status = succeeded4680worker2 # [ 66.397647] nix-daemon[858]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4681worker2 # [ 66.429455] nix-daemon[858]: reaped child process 1093, status = succeeded4682lb # [ 66.385852] niks3-server[1013]: 2026/09/23 05:13:42 INFO Received uploads request method=POST path=/api/pending_closures4683worker2 # [ 66.444526] nix-grpc-daemon[873]: time=2026-09-23T05:13:43.007Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4684worker2 # [ 66.446096] nix-grpc-daemon[873]: time=2026-09-23T05:13:43.007Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4685lb # [ 66.480534] niks3-server[1013]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4686lb # [ 66.492617] niks3-server[1013]: 2026/09/23 05:13:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4687worker2 # [ 66.540165] nix-grpc-daemon[873]: time=2026-09-23T05:13:43.102Z level=INFO msg="Uploading 1 narinfos"4688lb # [ 66.501097] niks3-server[1013]: 2026/09/23 05:13:43 INFO Signed narinfos id=16 count=14689lb # [ 66.507831] niks3-server[1013]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4690lb # [ 66.513990] niks3-server[1013]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4691lb # [ 66.525682] niks3-server[1013]: 2026/09/23 05:13:43 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4692lb # [ 66.534401] niks3-server[1013]: 2026/09/23 05:13:43 INFO Completed upload id=164693worker2 # [ 66.577112] nix-grpc-daemon[873]: time=2026-09-23T05:13:43.141Z level=INFO msg="Upload complete. (328ms)"4694lb # [ 66.539662] niks3-server[1013]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4695worker2 # [ 66.597607] nix-grpc-daemon[771]: ts=2026-09-23T05:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14696worker2 # [ 66.624456] nix-grpc-daemon[771]: ts=2026-09-23T05:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14697worker2 # [ 66.632779] nix-daemon[858]: reaped child process 1056, status = succeeded4698client # [ 66.645801] systemd[1]: dedup1.service: Deactivated successfully.4699client # [ 66.653989] systemd[1]: dedup1.service: Consumed 189ms CPU time over 5.123s wall clock time, 41.5M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.4700client # [ 66.684694] systemd[1]: dedup2.service: Deactivated successfully.4701client # [ 66.687239] systemd[1]: dedup2.service: Consumed 208ms CPU time over 4.059s wall clock time, 33.2M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4702client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.18 seconds)4703client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4704client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4705worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4706worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4707worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4708worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4709worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4710worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4711worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4712worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4713(finished: subtest: two clients wanting the same drv build it once, in 6.26 seconds)4714subtest: --timeout and --max-silent-time reach the builder4715worker1 # [ 67.985289] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376404716worker2 # [ 68.007759] nix-grpc-daemon[771]: ts=2026-09-23T05:13:44Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04717worker2 # [ 68.014887] nix-grpc-daemon[771]: ts=2026-09-23T05:13:44Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14718worker1 # [ 68.020495] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404719worker1 # [ 68.042855] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376404720worker1 # [ 68.061014] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376404721worker1 # [ 68.078553] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4722worker1 # [ 68.250451] nix-daemon[835]: accepted connection from pid 986, user nix-grpc-daemon (trusted)4723worker1 # [ 68.275398] nix-daemon[835]: reaped child process 989, status = succeeded4724lb # [ 68.238867] niks3-server[1013]: 2026/09/23 05:13:44 INFO Received uploads request method=POST path=/api/pending_closures4725worker1 # [ 68.292965] nix-grpc-daemon[862]: time=2026-09-23T05:13:44.848Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4726worker1 # [ 68.294578] nix-grpc-daemon[862]: time=2026-09-23T05:13:44.848Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4727worker1 # [ 68.332695] nix-grpc-daemon[862]: time=2026-09-23T05:13:44.888Z level=INFO msg="Uploading 1 narinfos"4728lb # [ 68.276540] niks3-server[1013]: 2026/09/23 05:13:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4729lb # [ 68.294457] niks3-server[1013]: 2026/09/23 05:13:44 INFO Signed narinfos id=17 count=14730lb # [ 68.302828] niks3-server[1013]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4731worker1 # [ 68.361699] nix-grpc-daemon[862]: time=2026-09-23T05:13:44.917Z level=INFO msg="Upload complete. (262ms)"4732lb # [ 68.319540] niks3-server[1013]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4733lb # [ 68.329683] niks3-server[1013]: 2026/09/23 05:13:44 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4734worker1 # [ 68.373020] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=4244735lb # [ 68.334407] niks3-server[1013]: 2026/09/23 05:13:44 INFO Completed upload id=174736lb # [ 68.337014] niks3-server[1013]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4737worker1 # [ 68.384985] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404738worker1 # [ 68.399792] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4739worker1 # [ 68.406269] nix-daemon[835]: reaped child process 984, status = succeeded4740lb # [ 68.366595] envoy[900]: conn peer=[2001:db8:1::1]:57748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9541 tx=4369 ms=67294741lb # [ 68.371928] envoy[900]: conn peer=[2001:db8:1::1]:34360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1994742worker1 # [ 68.421349] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4743lb # [ 68.377838] envoy[900]: conn peer=[2001:db8:1::1]:34388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=654744lb # [ 68.382794] envoy[900]: conn peer=[2001:db8:1::1]:34378 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=375 ms=864745lb # [ 68.387199] envoy[900]: conn peer=[2001:db8:1::1]:34370 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=1064746lb # [ 68.391176] envoy[900]: conn peer=[2001:db8:1::1]:34394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1541 ms=48904747lb # [ 68.395000] envoy[900]: conn peer=[2001:db8:1::1]:34396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1198 ms=38014748worker1 # [ 74.438971] nix-grpc-daemon[768]: ts=2026-09-23T05:13:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04749worker1 # [ 74.477304] nix-daemon[835]: reaped child process 991, status = succeeded4750worker1 # [ 74.479955] nix-daemon[835]: reaped child process 993, status = succeeded4751worker1 # [ 74.686808] nix-grpc-daemon[768]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376404752worker2 # [ 74.705460] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04753worker2 # [ 74.714113] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14754worker2 # [ 74.724497] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804755worker2 # [ 74.742000] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454804756worker2 # [ 74.759323] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454804757worker2 # [ 74.776153] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4758worker2 # [ 74.946590] nix-daemon[858]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4759worker2 # [ 74.970162] nix-daemon[858]: reaped child process 1113, status = succeeded4760lb # [ 74.926529] niks3-server[1013]: 2026/09/23 05:13:51 INFO Received uploads request method=POST path=/api/pending_closures4761worker2 # [ 74.982503] nix-grpc-daemon[873]: time=2026-09-23T05:13:51.545Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4762worker2 # [ 74.984166] nix-grpc-daemon[873]: time=2026-09-23T05:13:51.545Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4763worker2 # [ 75.019963] nix-grpc-daemon[873]: time=2026-09-23T05:13:51.583Z level=INFO msg="Uploading 1 narinfos"4764lb # [ 74.967300] niks3-server[1013]: 2026/09/23 05:13:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4765lb # [ 74.983012] niks3-server[1013]: 2026/09/23 05:13:51 INFO Signed narinfos id=18 count=14766lb # [ 74.991932] niks3-server[1013]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4767lb # [ 75.001639] niks3-server[1013]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4768lb # [ 75.014225] niks3-server[1013]: 2026/09/23 05:13:51 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4769worker2 # [ 75.062042] nix-grpc-daemon[873]: time=2026-09-23T05:13:51.625Z level=INFO msg="Upload complete. (266ms)"4770lb # [ 75.021726] niks3-server[1013]: 2026/09/23 05:13:51 INFO Completed upload id=184771lb # [ 75.024696] niks3-server[1013]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4772worker2 # [ 75.072406] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=4244773worker2 # [ 75.093044] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804774worker2 # [ 75.104810] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4775worker2 # [ 75.108244] nix-daemon[858]: reaped child process 1108, status = succeeded4776worker2 # [ 75.123659] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4777lb # [ 78.366740] envoy[900]: conn peer=[2001:db8:1::1]:34406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3014 tx=953 ms=64754778worker2 # [ 80.945955] nix-grpc-daemon[771]: ts=2026-09-23T05:13:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04779worker2 # [ 80.986738] nix-daemon[858]: reaped child process 1115, status = succeeded4780worker2 # [ 80.993044] nix-daemon[858]: reaped child process 1117, status = succeeded4781(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.44 seconds)4782subtest: the first client leaving does not fail the second4783client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4784client # [ 81.249947] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4785client # Running as unit: share1.service; invocation ID: bbf232037aa442619bdfc1253d113f3c4786client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4787client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4788client # [ 81.474623] nix[1080]: don't know how to build these paths:4789client # [ 81.475740] nix[1080]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4790worker1 # [ 81.473989] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376404791worker2 # [ 81.486345] nix-grpc-daemon[771]: ts=2026-09-23T05:13:58Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04792worker1 # [ 81.490207] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404793worker2 # [ 81.492930] nix-grpc-daemon[771]: ts=2026-09-23T05:13:58Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14794worker1 # [ 81.507083] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376404795client # [ 81.520690] nix[1080]: copying 1 paths to 'lb:50051'...4796client # [ 81.524136] nix[1080]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4797worker1 # [ 81.524312] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376404798worker1 # [ 81.539912] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4799worker1 # [ 81.704480] nix-daemon[835]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)4800worker1 # [ 81.724284] nix-daemon[835]: reaped child process 1019, status = succeeded4801lb # [ 81.690156] niks3-server[1013]: 2026/09/23 05:13:58 INFO Received uploads request method=POST path=/api/pending_closures4802worker1 # [ 81.743089] nix-grpc-daemon[862]: time=2026-09-23T05:13:58.298Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4803worker1 # [ 81.745270] nix-grpc-daemon[862]: time=2026-09-23T05:13:58.298Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4804lb # [ 81.732906] niks3-server[1013]: 2026/09/23 05:13:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4805worker1 # [ 81.788312] nix-grpc-daemon[862]: time=2026-09-23T05:13:58.344Z level=INFO msg="Uploading 1 narinfos"4806lb # [ 81.748089] niks3-server[1013]: 2026/09/23 05:13:58 INFO Signed narinfos id=19 count=14807lb # [ 81.750648] niks3-server[1013]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4808lb # [ 81.764787] niks3-server[1013]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4809lb # [ 81.767627] niks3-server[1013]: 2026/09/23 05:13:58 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4810worker1 # [ 81.807296] nix-grpc-daemon[862]: time=2026-09-23T05:13:58.363Z level=INFO msg="Upload complete. (249ms)"4811lb # [ 81.774000] niks3-server[1013]: 2026/09/23 05:13:58 INFO Completed upload id=194812lb # [ 81.777726] niks3-server[1013]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4813worker1 # [ 81.816611] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=4324814worker1 # [ 81.834847] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404815worker1 # [ 81.841768] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4816worker1 # [ 81.846938] nix-daemon[835]: reaped child process 1014, status = succeeded4817worker1 # [ 81.860182] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)4818client # [ 81.859309] nix[1080]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4819client # [ 82.364979] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4820client # Running as unit: share2.service; invocation ID: 3404ed2868e1421f9145ecc3688777764821client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds)4822worker2 # [ 82.587817] nix-grpc-daemon[771]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:454804823client # [ 82.595992] nix[1096]: don't know how to build these paths:4824client # [ 82.598071] nix[1096]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4825worker2 # [ 82.605854] nix-grpc-daemon[771]: ts=2026-09-23T05:13:59Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04826worker1 # [ 82.611593] nix-grpc-daemon[768]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376404827client # [ 82.620560] nix[1096]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4828client: must succeed: sleep 2; systemctl kill -s INT share14829client # [ 85.746066] systemd[1]: share1.service: Sent signal SIGINT to main process 1080 (nix) on client request.4830client # [ 85.757094] nix[1080]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4831client # [ 85.771259] nix[1080]: error: interrupted by the user4832client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4833client: waiting for success: ! systemctl is-active share14834client # [ 85.839854] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4835client # [ 85.842733] systemd[1]: share1.service: Failed with result 'exit-code'.4836client # [ 85.845992] systemd[1]: share1.service: Consumed 189ms CPU time over 4.593s wall clock time, 39.4M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic.4837client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4838client: waiting for success: ! systemctl is-active share24839worker1 # [ 87.386270] nix-daemon[835]: reaped child process 1023, status = succeeded4840worker1 # [ 87.461573] nix-daemon[835]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)4841worker1 # [ 87.482089] nix-daemon[835]: reaped child process 1060, status = succeeded4842lb # [ 87.445906] niks3-server[1013]: 2026/09/23 05:14:04 INFO Received uploads request method=POST path=/api/pending_closures4843worker1 # [ 87.502736] nix-grpc-daemon[862]: time=2026-09-23T05:14:04.055Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4844worker1 # [ 87.504519] nix-grpc-daemon[862]: time=2026-09-23T05:14:04.056Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4845lb # [ 87.490588] niks3-server[1013]: 2026/09/23 05:14:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4846worker1 # [ 87.542350] nix-grpc-daemon[862]: time=2026-09-23T05:14:04.098Z level=INFO msg="Uploading 1 narinfos"4847lb # [ 87.505740] niks3-server[1013]: 2026/09/23 05:14:04 INFO Signed narinfos id=20 count=14848lb # [ 87.515938] niks3-server[1013]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4849lb # [ 87.529677] niks3-server[1013]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4850lb # [ 87.538997] niks3-server[1013]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4851worker1 # [ 87.581336] nix-grpc-daemon[862]: time=2026-09-23T05:14:04.137Z level=INFO msg="Upload complete. (294ms)"4852lb # [ 87.544598] niks3-server[1013]: 2026/09/23 05:14:04 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4853lb # [ 87.550082] niks3-server[1013]: 2026/09/23 05:14:04 INFO Completed upload id=204854lb # [ 87.551775] niks3-server[1013]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4855worker1 # [ 87.597164] nix-grpc-daemon[768]: ts=2026-09-23T05:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14856worker1 # [ 87.611912] nix-grpc-daemon[768]: ts=2026-09-23T05:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14857worker1 # [ 87.617753] nix-daemon[835]: reaped child process 1021, status = succeeded4858client # [ 87.670564] systemd[1]: share2.service: Deactivated successfully.4859client # [ 87.675730] systemd[1]: share2.service: Consumed 201ms CPU time over 5.307s wall clock time, 33.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4860client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4861client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4862client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds)4863(finished: subtest: the first client leaving does not fail the second, in 7.16 seconds)4864subtest: build hook: nix-daemon with builders = grpc://lb4865client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4866lb # [ 88.372441] envoy[900]: conn peer=[2001:db8:1::1]:47710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3028 tx=953 ms=63204867lb # [ 88.372680] envoy[900]: conn peer=[2001:db8:1::1]:58040 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3003 tx=811 ms=43054868lb # [ 88.372846] envoy[900]: conn peer=[2001:db8:1::1]:58052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1198 ms=50624869client # [ 88.482364] systemd[1]: Started Nix Daemon.4870client # [ 88.605389] nix-daemon[1148]: accepted connection from pid 1145, user root (trusted)4871worker2 # [ 88.759528] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:454804872worker1 # [ 88.763900] nix-grpc-daemon[768]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376404873worker2 # [ 88.782937] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04874worker2 # [ 88.790574] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14875worker2 # [ 88.801367] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804876worker2 # [ 88.812697] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454804877worker2 # [ 88.826924] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454804878worker2 # [ 88.842109] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4879worker2 # [ 88.999142] nix-daemon[858]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)4880worker2 # [ 89.020388] nix-daemon[858]: reaped child process 1165, status = succeeded4881lb # [ 88.982358] niks3-server[1013]: 2026/09/23 05:14:05 INFO Received uploads request method=POST path=/api/pending_closures4882worker2 # [ 89.034885] nix-grpc-daemon[873]: time=2026-09-23T05:14:05.598Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4883worker2 # [ 89.036677] nix-grpc-daemon[873]: time=2026-09-23T05:14:05.598Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4884lb # [ 89.016172] niks3-server[1013]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4885worker2 # [ 89.067010] nix-grpc-daemon[873]: time=2026-09-23T05:14:05.630Z level=INFO msg="Uploading 1 narinfos"4886lb # [ 89.030185] niks3-server[1013]: 2026/09/23 05:14:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4887lb # [ 89.040511] niks3-server[1013]: 2026/09/23 05:14:05 INFO Signed narinfos id=21 count=14888lb # [ 89.043079] niks3-server[1013]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4889worker2 # [ 89.098104] nix-grpc-daemon[873]: time=2026-09-23T05:14:05.662Z level=INFO msg="Upload complete. (239ms)"4890lb # [ 89.056571] niks3-server[1013]: 2026/09/23 05:14:05 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4891lb # [ 89.060221] niks3-server[1013]: 2026/09/23 05:14:05 INFO Completed upload id=214892lb # [ 89.063012] niks3-server[1013]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4893worker2 # [ 89.108357] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=5124894worker2 # [ 89.124350] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804895worker2 # [ 89.131533] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4896worker2 # [ 89.137846] nix-daemon[858]: reaped child process 1160, status = succeeded4897worker2 # [ 89.151145] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4898worker2 # [ 90.422837] nix-daemon[858]: reaped child process 1169, status = succeeded4899worker2 # [ 90.552891] nix-daemon[858]: accepted connection from pid 1185, user nix-grpc-daemon (trusted)4900worker2 # [ 90.572630] nix-daemon[858]: reaped child process 1188, status = succeeded4901lb # [ 90.538137] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received uploads request method=POST path=/api/pending_closures4902worker2 # [ 90.591978] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.152Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4903worker2 # [ 90.593997] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.152Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4904lb # [ 90.577083] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4905worker2 # [ 90.626723] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.190Z level=INFO msg="Uploading 1 narinfos"4906lb # [ 90.591993] niks3-server[1013]: 2026/09/23 05:14:07 INFO Signed narinfos id=22 count=14907lb # [ 90.598619] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4908lb # [ 90.611583] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4909lb # [ 90.617350] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4910lb # [ 90.620717] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4911worker2 # [ 90.666119] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.229Z level=INFO msg="Upload complete. (298ms)"4912lb # [ 90.627900] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4913lb # [ 90.631495] niks3-server[1013]: 2026/09/23 05:14:07 INFO Completed upload id=224914worker2 # [ 90.678177] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14915worker2 # [ 90.697604] nix-daemon[858]: reaped child process 1167, status = succeeded4916worker1 # [ 90.725225] nix-grpc-daemon[768]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376404917worker1 # [ 90.741082] nix-grpc-daemon[768]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4918worker2 # [ 90.782807] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:454804919worker2 # [ 90.790139] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=1284920worker1 # [ 91.034342] nix-grpc-daemon[768]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376404921worker1 # [ 91.042770] nix-grpc-daemon[768]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376404922worker1 # [ 91.050624] nix-grpc-daemon[862]: time=2026-09-23T05:14:07.604Z level=INFO msg="All 1 paths already cached"4923worker2 # [ 91.062468] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04924worker2 # [ 91.070128] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14925worker2 # [ 91.079139] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804926worker2 # [ 91.090861] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454804927worker2 # [ 91.105803] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.669Z level=INFO msg="All 1 paths already cached"4928worker2 # [ 91.115898] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454804929worker2 # [ 91.130318] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4930worker2 # [ 91.284340] nix-daemon[858]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)4931worker2 # [ 91.307624] nix-daemon[858]: reaped child process 1197, status = succeeded4932lb # [ 91.268751] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received uploads request method=POST path=/api/pending_closures4933worker2 # [ 91.320757] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.884Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4934worker2 # [ 91.322368] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.884Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4935lb # [ 91.300384] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4936worker2 # [ 91.354420] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.918Z level=INFO msg="Uploading 1 narinfos"4937lb # [ 91.313785] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4938lb # [ 91.325062] niks3-server[1013]: 2026/09/23 05:14:07 INFO Signed narinfos id=23 count=14939lb # [ 91.327777] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4940lb # [ 91.337368] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4941lb # [ 91.347141] niks3-server[1013]: 2026/09/23 05:14:07 INFO Completed upload id=234942worker2 # [ 91.388917] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.952Z level=INFO msg="Upload complete. (241ms)"4943lb # [ 91.350358] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4944worker2 # [ 91.400117] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=6244945worker2 # [ 91.408303] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804946worker2 # [ 91.420966] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4947worker2 # [ 91.425988] nix-daemon[858]: reaped child process 1192, status = succeeded4948worker2 # [ 91.439620] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4949worker2 # [ 92.528581] nix-daemon[858]: reaped child process 1201, status = succeeded4950worker2 # [ 92.622243] nix-daemon[858]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)4951worker2 # [ 92.643421] nix-daemon[858]: reaped child process 1221, status = succeeded4952lb # [ 92.599815] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received uploads request method=POST path=/api/pending_closures4953worker2 # [ 92.652489] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.215Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4954worker2 # [ 92.654037] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.215Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4955lb # [ 92.648828] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4956worker2 # [ 92.704993] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.268Z level=INFO msg="Uploading 1 narinfos"4957lb # [ 92.664960] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4958lb # [ 92.674302] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4959lb # [ 92.680778] niks3-server[1013]: 2026/09/23 05:14:09 INFO Signed narinfos id=24 count=14960worker2 # [ 92.731382] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.295Z level=INFO msg="Upload complete. (281ms)"4961lb # [ 92.689890] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4962lb # [ 92.698871] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4963lb # [ 92.704070] niks3-server[1013]: 2026/09/23 05:14:09 INFO Completed upload id=244964worker2 # [ 92.745279] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14965worker1 # [ 92.753727] nix-grpc-daemon[768]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376404966lb # [ 92.708310] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4967worker1 # [ 92.761861] nix-grpc-daemon[768]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4968worker2 # [ 92.766213] nix-daemon[858]: reaped child process 1199, status = succeeded4969worker2 # [ 92.799662] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:454804970worker2 # [ 92.807664] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=1764971worker1 # [ 93.044215] nix-grpc-daemon[768]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376404972worker2 # [ 93.047851] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454804973worker2 # [ 93.058567] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.622Z level=INFO msg="All 2 paths already cached"4974worker2 # [ 93.080281] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04975worker2 # [ 93.087920] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14976worker2 # [ 93.096868] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804977worker2 # [ 93.108312] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454804978worker2 # [ 93.122985] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.686Z level=INFO msg="All 2 paths already cached"4979worker2 # [ 93.133686] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454804980worker2 # [ 93.147969] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4981worker2 # [ 93.320941] nix-daemon[858]: accepted connection from pid 1227, user nix-grpc-daemon (trusted)4982worker2 # [ 93.341690] nix-daemon[858]: reaped child process 1230, status = succeeded4983lb # [ 93.307044] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received uploads request method=POST path=/api/pending_closures4984worker2 # [ 93.359189] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.922Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4985worker2 # [ 93.360967] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.922Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4986worker2 # [ 93.387415] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.951Z level=INFO msg="Uploading 1 narinfos"4987lb # [ 93.338307] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4988lb # [ 93.355784] niks3-server[1013]: 2026/09/23 05:14:09 INFO Signed narinfos id=25 count=14989lb # [ 93.363099] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4990lb # [ 93.373214] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4991lb # [ 93.381344] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4992worker2 # [ 93.431168] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.995Z level=INFO msg="Upload complete. (264ms)"4993lb # [ 93.391051] niks3-server[1013]: 2026/09/23 05:14:09 INFO Completed upload id=254994lb # [ 93.396774] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4995worker2 # [ 93.440307] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=7524996worker2 # [ 93.456424] nix-grpc-daemon[771]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454804997worker2 # [ 93.466434] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4998worker2 # [ 93.472341] nix-daemon[858]: reaped child process 1225, status = succeeded4999worker2 # [ 93.488849] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5000worker2 # [ 94.720787] nix-daemon[858]: reaped child process 1234, status = succeeded5001worker2 # [ 94.817903] nix-daemon[858]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)5002worker2 # [ 94.839934] nix-daemon[858]: reaped child process 1253, status = succeeded5003lb # [ 94.802330] niks3-server[1013]: 2026/09/23 05:14:11 INFO Received uploads request method=POST path=/api/pending_closures5004worker2 # [ 94.856208] nix-grpc-daemon[873]: time=2026-09-23T05:14:11.417Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5005worker2 # [ 94.857764] nix-grpc-daemon[873]: time=2026-09-23T05:14:11.418Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"5006lb # [ 94.848662] niks3-server[1013]: 2026/09/23 05:14:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5007worker2 # [ 94.899415] nix-grpc-daemon[873]: time=2026-09-23T05:14:11.463Z level=INFO msg="Uploading 1 narinfos"5008lb # [ 94.860568] niks3-server[1013]: 2026/09/23 05:14:11 INFO Signed narinfos id=26 count=15009lb # [ 94.868422] niks3-server[1013]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst5010lb # [ 94.879290] niks3-server[1013]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls5011worker2 # [ 94.931816] nix-grpc-daemon[873]: time=2026-09-23T05:14:11.495Z level=INFO msg="Upload complete. (232ms)"5012lb # [ 94.890860] niks3-server[1013]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv5013lb # [ 94.899221] niks3-server[1013]: 2026/09/23 05:14:11 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5014lb # [ 94.903903] niks3-server[1013]: 2026/09/23 05:14:11 INFO Completed upload id=265015worker2 # [ 94.942628] nix-grpc-daemon[771]: ts=2026-09-23T05:14:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=15016lb # [ 94.908254] niks3-server[1013]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo5017worker2 # [ 94.963388] nix-grpc-daemon[771]: ts=2026-09-23T05:14:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:454805018worker2 # [ 94.968001] nix-daemon[858]: reaped child process 1232, status = succeeded5019worker1 # [ 94.978971] nix-grpc-daemon[768]: ts=2026-09-23T05:14:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376405020worker1 # [ 94.993275] nix-grpc-daemon[768]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...5021worker1 # [ 95.010183] nix-grpc-daemon[768]: ts=2026-09-23T05:14:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_out=2325022client # [ 95.135161] nix-daemon[1148]: reaped child process 1150, status = succeeded5023client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.82 seconds)5024client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top5025client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)5026client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5027client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)5028client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5029client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5030(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.94 seconds)5031subtest: an input only one worker has still reaches the builder5032client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only5033client # this derivation will be built:5034client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv5035client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...5036client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)5037client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure5038client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.13 seconds)5039worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5040worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)5041worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only5042worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)5043client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&25044client # [ 96.111297] nix-daemon[1148]: accepted connection from pid 1230, user root (trusted)5045client # this derivation will be built:5046client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5047client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5048worker1 # [ 96.263882] nix-grpc-daemon[768]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376405049worker2 # [ 96.267166] nix-grpc-daemon[771]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454805050client # copying 1 paths...5051client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...5052worker1 # [ 96.283367] nix-grpc-daemon[768]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376405053worker1 # [ 96.298624] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5054worker1 # [ 96.467905] nix-daemon[835]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5055worker1 # [ 96.491188] nix-daemon[835]: reaped child process 1075, status = succeeded5056lb # [ 96.450900] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures5057worker1 # [ 96.505132] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.060Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5058worker1 # [ 96.506754] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.060Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"5059lb # [ 96.488309] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5060worker1 # [ 96.544276] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.100Z level=INFO msg="Uploading 1 narinfos"5061lb # [ 96.503018] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5062lb # [ 96.512303] niks3-server[1013]: 2026/09/23 05:14:13 INFO Signed narinfos id=27 count=15063lb # [ 96.519005] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls5064lb # [ 96.526832] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5065lb # [ 96.541194] niks3-server[1013]: 2026/09/23 05:14:13 INFO Completed upload id=275066worker1 # [ 96.583838] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.140Z level=INFO msg="Upload complete. (269ms)"5067lb # [ 96.546755] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo5068worker1 # [ 96.596358] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=1205069worker2 # [ 96.606922] nix-grpc-daemon[771]: ts=2026-09-23T05:14:13Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=05070worker1 # [ 96.614577] nix-daemon[835]: reaped child process 1070, status = succeeded5071worker2 # [ 96.613550] nix-grpc-daemon[771]: ts=2026-09-23T05:14:13Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15072worker1 # [ 96.655339] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376405073worker1 # [ 96.671261] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376405074client # copying 2 paths to 'lb:50051'...5075client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...5076worker1 # [ 96.690337] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376405077client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...5078worker1 # [ 96.704147] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5079worker1 # [ 96.861640] nix-daemon[835]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)5080worker1 # [ 96.885766] nix-daemon[835]: reaped child process 1082, status = succeeded5081lb # [ 96.841579] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures5082lb # [ 96.850959] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures5083worker1 # [ 96.904111] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.458Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5084worker1 # [ 96.907266] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.458Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"5085worker1 # [ 96.909815] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.458Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"5086lb # [ 96.884314] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst5087lb # [ 96.902749] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5088lb # [ 96.914220] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst5089lb # [ 96.926574] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5090worker1 # [ 96.968710] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.523Z level=INFO msg="Uploading 3 narinfos"5091lb # [ 96.933624] niks3-server[1013]: 2026/09/23 05:14:13 INFO Signed narinfos id=28 count=15092lb # [ 96.935746] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5093lb # [ 96.944227] niks3-server[1013]: 2026/09/23 05:14:13 INFO Signed narinfos id=29 count=25094lb # [ 96.948921] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls5095lb # [ 96.961704] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5096lb # [ 96.972553] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5097lb # [ 96.981293] niks3-server[1013]: 2026/09/23 05:14:13 INFO Completed upload id=285098lb # [ 96.982926] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5099worker1 # [ 97.023970] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.580Z level=INFO msg="Upload complete. (300ms)"5100lb # [ 96.986078] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5101lb # [ 96.989778] niks3-server[1013]: 2026/09/23 05:14:13 INFO Completed upload id=295102lb # [ 96.994031] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5103worker1 # [ 97.035761] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=2 nar_bytes_in=9845104worker1 # [ 97.050867] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376405105worker1 # [ 97.065993] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5106worker1 # [ 97.073716] nix-daemon[835]: reaped child process 1077, status = succeeded5107client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5108worker1 # [ 97.089437] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5109client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5110client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5111worker1 # [ 98.388895] nix-daemon[835]: reaped child process 1086, status = succeeded5112lb # [ 98.373351] envoy[900]: conn peer=[2001:db8:1::1]:58066 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=295113worker1 # [ 98.482178] nix-daemon[835]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5114worker1 # [ 98.505814] nix-daemon[835]: reaped child process 1105, status = succeeded5115lb # [ 98.378692] envoy[900]: conn peer=[2001:db8:1::1]:58064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1944 ms=20505116lb # [ 98.378879] envoy[900]: conn peer=[2001:db8:1::1]:50684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2158 ms=17915117lb # [ 98.379017] envoy[900]: conn peer=[2001:db8:1::1]:50686 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=295118lb # [ 98.379135] envoy[900]: conn peer=[2001:db8:1::1]:50694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4378 tx=2390 ms=20235119lb # [ 98.379255] envoy[900]: conn peer=[2001:db8:1::1]:50696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=925120lb # [ 98.462735] niks3-server[1013]: 2026/09/23 05:14:15 INFO Received uploads request method=POST path=/api/pending_closures5121worker1 # [ 98.540648] nix-grpc-daemon[862]: time=2026-09-23T05:14:15.094Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5122lb # [ 98.525108] niks3-server[1013]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5123worker1 # [ 98.577578] nix-grpc-daemon[862]: time=2026-09-23T05:14:15.133Z level=INFO msg="Uploading 1 narinfos"5124lb # [ 98.535781] niks3-server[1013]: 2026/09/23 05:14:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5125lb # [ 98.544252] niks3-server[1013]: 2026/09/23 05:14:15 INFO Signed narinfos id=30 count=15126lb # [ 98.549118] niks3-server[1013]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5127worker1 # [ 98.601187] nix-grpc-daemon[862]: time=2026-09-23T05:14:15.157Z level=INFO msg="Upload complete. (293ms)"5128lb # [ 98.560916] niks3-server[1013]: 2026/09/23 05:14:15 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5129lb # [ 98.564147] niks3-server[1013]: 2026/09/23 05:14:15 INFO Completed upload id=305130lb # [ 98.567461] niks3-server[1013]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5131client # copying 1 paths...5132worker1 # [ 98.613183] nix-grpc-daemon[768]: ts=2026-09-23T05:14:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15133worker1 # [ 98.634667] nix-grpc-daemon[768]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376405134worker1 # [ 98.640575] nix-daemon[835]: reaped child process 1084, status = succeeded5135client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5136worker2 # [ 98.667087] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:454805137worker2 # [ 98.679707] nix-grpc-daemon[771]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...5138worker2 # [ 98.695345] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=1205139client # [ 98.821430] nix-daemon[1148]: reaped child process 1233, status = succeeded5140client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.91 seconds)5141client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25142client # [ 98.990472] nix-daemon[1148]: accepted connection from pid 1252, user root (trusted)5143client # this derivation will be built:5144client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5145worker1 # [ 99.140126] nix-grpc-daemon[768]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376405146client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5147worker2 # [ 99.148803] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454805148worker2 # [ 99.173877] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05149worker2 # [ 99.181680] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15150worker2 # [ 99.191046] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454805151worker2 # [ 99.201951] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454805152client # copying 1 paths to 'lb:50051'...5153client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5154worker2 # [ 99.220930] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454805155worker2 # [ 99.236165] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5156worker2 # [ 99.256113] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)5157worker2 # [ 99.289410] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 2/5)5158worker2 # [ 99.354392] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 3/5)5159worker2 # [ 99.665745] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 300 ms (attempt 4/5)5160worker2 # [ 99.970973] nix-grpc-daemon[771]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5161worker2 # [ 99.980291] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5162worker2 # [ 99.987855] nix-grpc-daemon[771]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5163worker2 # [ 100.154582] nix-daemon[858]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5164worker2 # [ 100.176293] nix-daemon[858]: reaped child process 1273, status = succeeded5165lb # [ 100.143408] niks3-server[1013]: 2026/09/23 05:14:16 INFO Received uploads request method=POST path=/api/pending_closures5166worker2 # [ 100.196478] nix-grpc-daemon[873]: time=2026-09-23T05:14:16.759Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5167worker2 # [ 100.198199] nix-grpc-daemon[873]: time=2026-09-23T05:14:16.759Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5168lb # [ 100.175512] niks3-server[1013]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5169lb # [ 100.190053] niks3-server[1013]: 2026/09/23 05:14:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5170worker2 # [ 100.241520] nix-grpc-daemon[873]: time=2026-09-23T05:14:16.805Z level=INFO msg="Uploading 1 narinfos"5171lb # [ 100.203894] niks3-server[1013]: 2026/09/23 05:14:16 INFO Signed narinfos id=31 count=15172lb # [ 100.211527] niks3-server[1013]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5173worker2 # [ 100.268059] nix-grpc-daemon[873]: time=2026-09-23T05:14:16.832Z level=INFO msg="Upload complete. (266ms)"5174lb # [ 100.226927] niks3-server[1013]: 2026/09/23 05:14:16 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5175lb # [ 100.235410] niks3-server[1013]: 2026/09/23 05:14:16 INFO Completed upload id=315176lb # [ 100.239478] niks3-server[1013]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5177worker2 # [ 100.280298] nix-grpc-daemon[771]: ts=2026-09-23T05:14:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 paths=1 nar_bytes_in=5685178worker2 # [ 100.295774] nix-grpc-daemon[771]: ts=2026-09-23T05:14:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454805179worker2 # [ 100.308470] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5180worker2 # [ 100.324002] nix-daemon[858]: reaped child process 1263, status = succeeded5181worker2 # [ 100.328402] nix-grpc-daemon[771]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...5182client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5183worker2 # [ 100.344958] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5184client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5185client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5186worker2 # [ 101.810804] nix-daemon[858]: reaped child process 1278, status = succeeded5187worker2 # [ 101.929389] nix-daemon[858]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)5188worker2 # [ 101.951484] nix-daemon[858]: reaped child process 1297, status = succeeded5189lb # [ 101.913664] niks3-server[1013]: 2026/09/23 05:14:18 INFO Received uploads request method=POST path=/api/pending_closures5190worker2 # [ 101.966042] nix-grpc-daemon[873]: time=2026-09-23T05:14:18.529Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5191lb # [ 101.944078] niks3-server[1013]: 2026/09/23 05:14:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5192worker2 # [ 102.002470] nix-grpc-daemon[873]: time=2026-09-23T05:14:18.566Z level=INFO msg="Uploading 1 narinfos"5193lb # [ 101.960230] niks3-server[1013]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5194lb # [ 101.975027] niks3-server[1013]: 2026/09/23 05:14:18 INFO Signed narinfos id=32 count=15195lb # [ 101.982967] niks3-server[1013]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5196worker2 # [ 102.037742] nix-grpc-daemon[873]: time=2026-09-23T05:14:18.601Z level=INFO msg="Upload complete. (280ms)"5197lb # [ 101.996428] niks3-server[1013]: 2026/09/23 05:14:18 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5198lb # [ 102.005820] niks3-server[1013]: 2026/09/23 05:14:18 INFO Completed upload id=325199lb # [ 102.010021] niks3-server[1013]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5200worker2 # [ 102.053294] nix-grpc-daemon[771]: ts=2026-09-23T05:14:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15201worker2 # [ 102.070479] nix-daemon[858]: reaped child process 1275, status = succeeded5202client # copying 1 paths...5203worker1 # [ 102.098518] nix-grpc-daemon[768]: ts=2026-09-23T05:14:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376405204worker1 # [ 102.112732] nix-grpc-daemon[768]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5205client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5206worker2 # [ 102.158829] nix-grpc-daemon[771]: ts=2026-09-23T05:14:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:454805207worker2 # [ 102.166119] nix-grpc-daemon[771]: ts=2026-09-23T05:14:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=1205208client # [ 102.252319] nix-daemon[1148]: reaped child process 1255, status = succeeded5209client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.43 seconds)5210client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25211client # [ 102.427273] nix-daemon[1148]: accepted connection from pid 1273, user root (trusted)5212client # this derivation will be built:5213client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5214worker1 # [ 102.583053] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376405215client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5216worker2 # [ 102.588409] nix-grpc-daemon[771]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454805217worker2 # [ 102.600615] nix-grpc-daemon[873]: time=2026-09-23T05:14:19.164Z level=INFO msg="All 1 paths already cached"5218worker2 # [ 102.617682] nix-grpc-daemon[771]: ts=2026-09-23T05:14:19Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05219worker2 # [ 102.624610] nix-grpc-daemon[771]: ts=2026-09-23T05:14:19Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15220worker1 # [ 102.627930] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376405221worker1 # [ 102.644925] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376405222worker1 # [ 102.662170] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.217Z level=INFO msg="All 1 paths already cached"5223client # copying 1 paths to 'lb:50051'...5224client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5225worker1 # [ 102.674506] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376405226worker1 # [ 102.690770] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5227worker1 # [ 102.854815] nix-daemon[835]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)5228worker1 # [ 102.876429] nix-daemon[835]: reaped child process 1113, status = succeeded5229lb # [ 102.838467] niks3-server[1013]: 2026/09/23 05:14:19 INFO Received uploads request method=POST path=/api/pending_closures5230worker1 # [ 102.893822] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.449Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5231worker1 # [ 102.895535] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.449Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5232lb # [ 102.873472] niks3-server[1013]: 2026/09/23 05:14:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5233worker1 # [ 102.921941] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.478Z level=INFO msg="Uploading 1 narinfos"5234lb # [ 102.886998] niks3-server[1013]: 2026/09/23 05:14:19 INFO Signed narinfos id=33 count=15235lb # [ 102.891196] niks3-server[1013]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5236lb # [ 102.905326] niks3-server[1013]: 2026/09/23 05:14:19 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5237worker1 # [ 102.960091] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.516Z level=INFO msg="Upload complete. (249ms)"5238lb # [ 102.918845] niks3-server[1013]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5239lb # [ 102.925709] niks3-server[1013]: 2026/09/23 05:14:19 INFO Completed upload id=335240lb # [ 102.930167] niks3-server[1013]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5241worker1 # [ 102.972622] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=5685242worker1 # [ 102.988349] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376405243worker1 # [ 102.996000] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5244worker1 # [ 103.001993] nix-daemon[835]: reaped child process 1108, status = succeeded5245worker1 # [ 103.014588] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted)5246client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5247client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5248client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5249worker1 # [ 104.064624] nix-daemon[835]: reaped child process 1117, status = succeeded5250worker1 # [ 104.151598] nix-daemon[835]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5251worker1 # [ 104.170131] nix-daemon[835]: reaped child process 1136, status = succeeded5252lb # [ 104.135322] niks3-server[1013]: 2026/09/23 05:14:20 INFO Received uploads request method=POST path=/api/pending_closures5253worker1 # [ 104.188399] nix-grpc-daemon[862]: time=2026-09-23T05:14:20.743Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5254lb # [ 104.164801] niks3-server[1013]: 2026/09/23 05:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5255worker1 # [ 104.215700] nix-grpc-daemon[862]: time=2026-09-23T05:14:20.771Z level=INFO msg="Uploading 1 narinfos"5256lb # [ 104.177938] niks3-server[1013]: 2026/09/23 05:14:20 INFO Signed narinfos id=34 count=15257lb # [ 104.189326] niks3-server[1013]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5258lb # [ 104.199467] niks3-server[1013]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5259lb # [ 104.213395] niks3-server[1013]: 2026/09/23 05:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5260worker1 # [ 104.262574] nix-grpc-daemon[862]: time=2026-09-23T05:14:20.818Z level=INFO msg="Upload complete. (234ms)"5261lb # [ 104.221784] niks3-server[1013]: 2026/09/23 05:14:20 INFO Completed upload id=345262lb # [ 104.224466] niks3-server[1013]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5263client # copying 1 paths...5264worker1 # [ 104.273003] nix-grpc-daemon[768]: ts=2026-09-23T05:14:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15265worker1 # [ 104.289091] nix-grpc-daemon[768]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376405266worker1 # [ 104.294530] nix-daemon[835]: reaped child process 1115, status = succeeded5267client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5268worker2 # [ 104.323962] nix-grpc-daemon[771]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:454805269worker2 # [ 104.334521] nix-grpc-daemon[771]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5270worker2 # [ 104.353910] nix-grpc-daemon[771]: ts=2026-09-23T05:14:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=1205271client # [ 104.480433] nix-daemon[1148]: reaped child process 1276, status = succeeded5272client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.23 seconds)5273(finished: subtest: an input only one worker has still reaches the builder, in 9.22 seconds)5274subtest: upload whose reference the worker lacks is completed from the cache5275client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5276client # this derivation will be built:5277client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5278client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5279client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.40 seconds)5280client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5281client # this derivation will be built:5282client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5283client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5284client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.36 seconds)5285client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5286client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.14 seconds)5287worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5288worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.28 seconds)5289worker2 # [ 105.762997] systemd[1]: Stopping Nix worker-protocol over gRPC...5290worker2 # [ 105.818599] nix-grpc-daemon[771]: ts=2026-09-23T05:14:22Z level=info event=signal_exit5291lb # [ 106.762062] niks3-server[1013]: 2026/09/23 05:14:23 INFO lead: released remote=192.168.1.4:577265292worker2 # [ 107.803140] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=scheduler_restarting5293worker1 # [ 107.806195] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_restarting addr=lb:500515294worker2 # [ 107.821453] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:454765295worker2 # [ 107.827213] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=worker_gone addr=192.168.1.3:500515296worker2 # [ 107.833791] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=scheduler_restarting addr=lb:500515297worker2 # [ 107.838378] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:454765298worker2 # [ 107.846989] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=worker_gone addr=192.168.1.4:500515299worker2 # [ 107.882105] nix-daemon[858]: reaped child process 860, status = succeeded5300worker2 # [ 107.993552] nix-grpc-daemon[771]: E0000 00:00:1790140464.556532 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5301worker2 # [ 108.000517] nix-grpc-daemon[771]: E0000 00:00:1790140464.557346 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5302worker1 # [ 108.006922] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_connected addr=lb:500515303worker1 # [ 108.012816] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker2 # [ 108.048930] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5305worker1 # [ 108.213884] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_connected addr=lb:500515306worker1 # [ 108.219652] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307lb # [ 108.366799] envoy[900]: conn peer=[2001:db8:1::1]:50700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4257 tx=2071 ms=24925308lb # [ 108.371960] envoy[900]: conn peer=[2001:db8:1::1]:50704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=915309worker1 # [ 108.420574] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_connected addr=lb:500515310lb # [ 108.376831] envoy[900]: conn peer=[2001:db8:1::1]:34154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=275311lb # [ 108.383142] envoy[900]: conn peer=[2001:db8:1::1]:50720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2124 ms=30425312worker1 # [ 108.425804] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313lb # [ 108.388136] envoy[900]: conn peer=[2001:db8:1::1]:34160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3754 tx=2097 ms=18295314lb # [ 108.392836] envoy[900]: conn peer=[2001:db8:1::1]:34172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=955315lb # [ 108.397478] envoy[900]: conn peer=[2001:db8:1::4]:46050 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=76994 tx=4394 ms=891965316worker1 # [ 108.626752] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_connected addr=lb:500515317worker1 # [ 108.632068] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5318worker1 # [ 108.797063] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_take_over5319lb # [ 108.755104] niks3-server[1013]: 2026/09/23 05:14:25 INFO lead: acquired remote=192.168.1.3:419905320worker1 # [ 108.834749] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_connected addr=lb:500515321worker1 # [ 108.838837] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5322worker1 # [ 109.035221] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_connected addr=lb:500515323worker1 # [ 109.055742] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:456705324worker1 # [ 109.061611] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05325worker2 # [ 109.809729] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5326worker2 # [ 109.811739] systemd[1]: Stopped Nix worker-protocol over gRPC.5327worker2 # [ 109.820675] systemd[1]: nix-grpc-daemon.service: Consumed 3.858s CPU time over 1min 31.249s wall clock time, 71.2M memory peak, 117.4K incoming IP traffic, 298.3K outgoing IP traffic.5328worker2 # [ 109.831297] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5329worker2 # [ 109.842649] systemd[1]: Closed Nix worker-protocol over gRPC.5330worker2 # [ 109.850069] systemd[1]: nix-grpc-daemon.socket: Consumed 678us CPU time over 1min 39.570s wall clock time, 256K memory peak, 125K incoming IP traffic, 102.1K outgoing IP traffic.5331lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5332lb: must succeed: curl -sf localhost:9901/clusters5333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5334lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5335client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25336worker1 # [ 110.170000] nix-grpc-daemon[768]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376405337worker1 # [ 110.180276] nix-grpc-daemon[768]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376405338client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5339client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.27 seconds)5340worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5341worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5342worker2 # [ 110.307068] systemd[1]: Listening on Nix worker-protocol over gRPC.5343worker1 # [ 110.384468] systemd[1]: Stopping Nix worker-protocol over gRPC...5344worker1 # [ 110.716521] nix-grpc-daemon[768]: ts=2026-09-23T05:14:27Z level=info event=signal_exit5345lb # [ 110.761316] niks3-server[1013]: 2026/09/23 05:14:27 INFO lead: released remote=192.168.1.3:419905346worker2 # [ 110.806872] systemd[1]: Starting scheduler-token.service...5347worker2 # [ 110.897410] systemd[1]: scheduler-token.service: Deactivated successfully.5348worker2 # [ 110.905288] systemd[1]: Finished scheduler-token.service.5349worker2 # [ 110.907420] systemd[1]: scheduler-token.service: Consumed 59ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5350worker2 # [ 110.918583] systemd[1]: Starting Nix worker-protocol over gRPC...5351worker2 # [ 111.067209] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5352worker2 # [ 111.069579] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5353worker2 # [ 111.073438] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15354worker2 # [ 111.078901] nix-grpc-daemon[1324]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5355lb # [ 111.052393] niks3-server[1013]: 2026/09/23 05:14:27 INFO lead: acquired remote=192.168.1.4:350225356worker2 # [ 111.096133] nix-grpc-daemon[1324]: E0000 00:00:1790140467.642324 1324 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5357worker2 # [ 111.124139] nix-grpc-daemon[1324]: E0000 00:00:1790140467.646072 1324 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5358worker2 # [ 111.126418] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5359worker2 # [ 111.134134] nix-grpc-daemon[1324]: E0000 00:00:1790140467.658795 1366 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5360worker2 # [ 111.137047] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_take_over5361worker2 # [ 111.138321] nix-grpc-daemon[1324]: E0000 00:00:1790140467.678282 1366 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5362worker2 # [ 111.140576] nix-grpc-daemon[1324]: E0000 00:00:1790140467.678415 1366 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5363worker2 # [ 111.145787] systemd[1]: Started Nix worker-protocol over gRPC.5364worker2 # [ 111.148824] nix-grpc-daemon[1324]: E0000 00:00:1790140467.697406 1366 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5365worker2 # [ 111.151086] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_connected addr=lb:500515366worker2 # [ 111.152348] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367worker2 # [ 111.350123] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_connected addr=lb:500515368worker2 # [ 111.355332] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5369worker2 # [ 111.556410] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_connected addr=lb:500515370worker2 # [ 111.561578] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5371worker2 # [ 111.762547] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_connected addr=lb:500515372worker2 # [ 111.767673] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5373worker1 # [ 111.811389] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=scheduler_restarting5374worker1 # [ 111.815387] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=scheduler_restarting addr=lb:500515375worker1 # [ 111.819952] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:456705376worker1 # [ 111.825153] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=worker_gone addr=192.168.1.3:500515377worker1 # [ 111.856312] nix-daemon[835]: reaped child process 837, status = succeeded5378worker1 # [ 111.907535] nix-grpc-daemon[768]: E0000 00:00:1790140468.463548 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5379worker1 # [ 111.920702] nix-grpc-daemon[768]: E0000 00:00:1790140468.476894 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5380worker2 # [ 111.969072] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_connected addr=lb:500515381worker2 # [ 111.974508] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5382worker1 # [ 112.027805] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5383worker2 # [ 112.178516] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_connected addr=lb:500515384worker2 # [ 112.192133] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=oidc_jwks provider=mock keys=15385worker2 # [ 112.197942] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:483565386worker2 # [ 112.203418] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05387worker1 # [ 113.754621] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5388worker1 # [ 113.759411] systemd[1]: Stopped Nix worker-protocol over gRPC.5389worker1 # [ 113.765190] systemd[1]: nix-grpc-daemon.service: Consumed 3.008s CPU time over 1min 35.340s wall clock time, 60.6M memory peak, 92.5K incoming IP traffic, 164.3K outgoing IP traffic.5390worker1 # [ 113.782929] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5391worker1 # [ 113.797616] systemd[1]: Closed Nix worker-protocol over gRPC.5392lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5393worker1 # [ 113.799553] systemd[1]: nix-grpc-daemon.socket: Consumed 684us CPU time over 1min 43.583s wall clock time, 256K memory peak, 102.6K incoming IP traffic, 90.8K outgoing IP traffic.5394lb: must succeed: curl -sf localhost:9901/clusters5395lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5396lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5397client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25398worker2 # [ 114.098180] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483705399worker2 # [ 114.116435] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5400worker2 # [ 114.144983] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)5401worker2 # [ 114.201905] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 2/5)5402worker2 # [ 114.364343] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 3/5)5403worker2 # [ 114.451084] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 333 ms (attempt 4/5)5404worker2 # [ 114.786612] nix-grpc-daemon[1324]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5405worker2 # [ 114.793645] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5406client # copying 2 paths...5407client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5408worker2 # [ 114.820885] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483705409client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5410worker2 # [ 114.838942] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5411worker2 # [ 114.870608] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:31Z level=info event=niks3_push_started pid=13855412worker2 # [ 115.024561] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.588Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5413worker2 # [ 115.136067] nix-daemon[858]: accepted connection from pid 1398, user nix-grpc-daemon (trusted)5414worker2 # [ 115.153884] nix-daemon[858]: reaped child process 1401, status = succeeded5415lb # [ 115.117950] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received uploads request method=POST path=/api/pending_closures5416lb # [ 115.129190] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received uploads request method=POST path=/api/pending_closures5417worker2 # [ 115.182411] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.744Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5418worker2 # [ 115.183871] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.744Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5419worker2 # [ 115.187148] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.744Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5420lb # [ 115.194956] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5421lb # [ 115.209174] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5422lb # [ 115.211784] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5423lb # [ 115.230472] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5424lb # [ 115.236712] niks3-server[1013]: 2026/09/23 05:14:31 INFO Signed narinfos id=35 count=15425worker2 # [ 115.279706] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.840Z level=INFO msg="Uploading 3 narinfos"5426lb # [ 115.242636] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5427lb # [ 115.247571] niks3-server[1013]: 2026/09/23 05:14:31 INFO Signed narinfos id=36 count=25428lb # [ 115.250353] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5429lb # [ 115.266246] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5430lb # [ 115.275428] niks3-server[1013]: 2026/09/23 05:14:31 INFO Completed upload id=355431lb # [ 115.278549] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5432lb # [ 115.282148] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5433lb # [ 115.289079] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5434lb # [ 115.291100] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5435lb # [ 115.294237] niks3-server[1013]: 2026/09/23 05:14:31 INFO Completed upload id=365436worker2 # [ 115.337048] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.900Z level=INFO msg="Upload complete. (308ms)"5437worker2 # [ 115.349228] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=0 paths=2 nar_bytes_in=3125438worker2 # [ 115.366624] nix-daemon[858]: reaped child process 1383, status = succeeded5439client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.51 seconds)5440worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5441worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5442worker1 # [ 115.489025] systemd[1]: Listening on Nix worker-protocol over gRPC.5443worker1 # [ 115.497612] systemd[1]: Starting Nix worker-protocol over gRPC...5444worker1 # [ 115.654437] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5445worker1 # [ 115.656791] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5446worker1 # [ 115.660911] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15447worker1 # [ 115.666332] nix-grpc-daemon[1159]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5448worker1 # [ 115.667756] nix-grpc-daemon[1159]: E0000 00:00:1790140472.222075 1159 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5449lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5450lb: must succeed: curl -sf localhost:9901/clusters5451worker1 # [ 115.696168] nix-grpc-daemon[1159]: E0000 00:00:1790140472.228096 1159 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5452worker1 # [ 115.698444] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5453worker1 # [ 115.717249] nix-grpc-daemon[1159]: E0000 00:00:1790140472.238216 1202 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5454worker1 # [ 115.720261] systemd[1]: Started Nix worker-protocol over gRPC.5455worker1 # [ 115.721084] nix-grpc-daemon[1159]: E0000 00:00:1790140472.258014 1202 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5456worker1 # [ 115.726412] nix-grpc-daemon[1159]: E0000 00:00:1790140472.258177 1202 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5457worker1 # [ 115.730728] nix-grpc-daemon[1159]: E0000 00:00:1790140472.272545 1202 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5458worker1 # [ 115.733254] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=scheduler_connected addr=lb:500515459worker2 # [ 115.732932] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:483565460worker2 # [ 115.738840] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05461lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5462lb: must succeed: curl -sf localhost:9901/clusters5463lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5464lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5465(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.33 seconds)5466subtest: a niks3 restart does not cost the leader its role5467lb: must succeed: curl -sf localhost:9901/clusters5468lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5469lb: must succeed: curl -sf localhost:9901/clusters5470lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5471lb # [ 116.911962] systemd[1]: Stopping niks3 server...5472lb # [ 116.917047] niks3-server[1013]: 2026/09/23 05:14:33 INFO Shutdown signal received, draining in-flight requests timeout=10s5473worker1 # [ 116.962107] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:33Z level=info event=scheduler_lock_lost status=2005474worker2 # [ 116.961958] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:33Z level=info event=scheduler_lock_lost status=2005475lb # [ 116.923457] niks3-server[1013]: 2026/09/23 05:14:33 INFO lead: released remote=192.168.1.4:350225476lb # [ 116.935067] systemd[1]: niks3.service: Deactivated successfully.5477lb # [ 116.938937] systemd[1]: Stopped niks3 server.5478lb # [ 116.939713] systemd[1]: niks3.service: Consumed 691ms CPU time over 1min 28.487s wall clock time, 27.7M memory peak, 98.4K incoming IP traffic, 142.1K outgoing IP traffic.5479lb # [ 116.950225] systemd[1]: Starting niks3 server...5480lb # [ 117.017093] niks3-server[1152]: 2026/09/23 05:14:33 goose: no migrations to run. current version: 202609200000005481lb # [ 117.024911] niks3-server[1152]: 2026/09/23 05:14:33 OK 1_commit_pending_closure.sql (5.99ms)5482lb # [ 117.028675] niks3-server[1152]: 2026/09/23 05:14:33 OK 2_object_stats_trigger.sql (3.38ms)5483lb # [ 117.030485] niks3-server[1152]: 2026/09/23 05:14:33 goose: up to current file version: 25484lb # [ 117.032079] niks3-server[1152]: 2026/09/23 05:14:33 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5485lb # [ 117.042989] niks3-server[1152]: 2026/09/23 05:14:33 INFO Read proxy enabled — serving cache objects from S35486lb # [ 117.048535] niks3-server[1152]: 2026/09/23 05:14:33 INFO Using socket-activated listener address=0.0.0.0:57515487lb # [ 117.049952] niks3-server[1152]: 2026/09/23 05:14:33 INFO systemd watchdog enabled interval=15s5488lb # [ 117.051262] niks3-server[1152]: 2026/09/23 05:14:33 INFO Starting HTTP server address=0.0.0.0:57515489lb # [ 117.054010] systemd[1]: Started niks3 server.5490lb: waiting for unit niks3.service5491lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5492lb: must succeed: curl -sf localhost:9901/clusters5493lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5494worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5495worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5496lb: must succeed: curl -sf localhost:9901/clusters5497lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5498lb: must succeed: curl -sf localhost:9901/clusters5499lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5500worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5501worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5502client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25503worker2 # [ 117.579267] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:483705504client # don't know how to build these paths:5505client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5506worker1 # [ 117.595474] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:371745507worker1 # [ 117.610612] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5508worker1 # [ 117.636291] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)5509worker1 # [ 117.717742] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 2/5)5510worker1 # [ 117.839659] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 310 ms (attempt 3/5)5511lb # [ 117.924858] niks3-server[1152]: 2026/09/23 05:14:34 INFO lead: acquired remote=192.168.1.4:350725512worker1 # [ 118.152697] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 4/5)5513worker1 # [ 118.198700] nix-grpc-daemon[1159]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5514worker1 # [ 118.205434] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5515worker2 # [ 118.252396] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05516worker2 # [ 118.257828] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15517worker2 # [ 118.297311] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483705518worker2 # [ 118.315376] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483705519client # copying 1 paths to 'lb:50051'...5520client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5521worker2 # [ 118.331230] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483705522worker2 # [ 118.346314] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5523lb # [ 118.372636] envoy[900]: conn peer=[2001:db8:1::1]:34178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=325524lb # [ 118.377647] envoy[900]: conn peer=[2001:db8:1::3]:45428 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=16090 tx=4442 ms=932845525lb # [ 118.382650] envoy[900]: conn peer=[2001:db8:1::1]:39538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=12675526worker2 # [ 118.499254] nix-daemon[858]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)5527worker2 # [ 118.521427] nix-daemon[858]: reaped child process 1423, status = succeeded5528lb # [ 118.477947] niks3-server[1152]: 2026/09/23 05:14:35 INFO Received uploads request method=POST path=/api/pending_closures5529worker2 # [ 118.535376] nix-grpc-daemon[1385]: time=2026-09-23T05:14:35.098Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5530worker2 # [ 118.537858] nix-grpc-daemon[1385]: time=2026-09-23T05:14:35.098Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5531lb # [ 118.519836] niks3-server[1152]: 2026/09/23 05:14:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5532lb # [ 118.552738] niks3-server[1152]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5533lb # [ 118.579964] niks3-server[1152]: 2026/09/23 05:14:35 INFO Signed narinfos id=37 count=15534lb # [ 118.582333] niks3-server[1152]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5535worker2 # [ 118.624141] nix-grpc-daemon[1385]: time=2026-09-23T05:14:35.186Z level=INFO msg="Uploading 1 narinfos"5536lb # [ 118.596105] niks3-server[1152]: 2026/09/23 05:14:35 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5537lb # [ 118.609815] niks3-server[1152]: 2026/09/23 05:14:35 INFO Completed upload id=375538worker2 # [ 118.652363] nix-grpc-daemon[1385]: time=2026-09-23T05:14:35.216Z level=INFO msg="Upload complete. (288ms)"5539lb # [ 118.617354] niks3-server[1152]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5540worker2 # [ 118.665946] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=0 paths=1 nar_bytes_in=5445541worker2 # [ 118.685118] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483705542worker2 # [ 118.697651] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5543worker2 # [ 118.701735] nix-daemon[858]: reaped child process 1418, status = succeeded5544client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5545client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5546worker2 # [ 118.717100] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5547client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5548client # farm-a> LOG-farm-a-after-niks3-restart5549worker2 # [ 119.989170] nix-daemon[858]: reaped child process 1427, status = succeeded5550worker2 # [ 120.103653] nix-daemon[858]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5551worker2 # [ 120.121959] nix-daemon[858]: reaped child process 1446, status = succeeded5552lb # [ 120.085636] niks3-server[1152]: 2026/09/23 05:14:36 INFO Received uploads request method=POST path=/api/pending_closures5553worker2 # [ 120.142618] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.704Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5554worker2 # [ 120.144423] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.704Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5555lb # [ 120.139125] niks3-server[1152]: 2026/09/23 05:14:36 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5556worker2 # [ 120.191203] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.754Z level=INFO msg="Uploading 1 narinfos"5557lb # [ 120.149565] niks3-server[1152]: 2026/09/23 05:14:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5558lb # [ 120.159617] niks3-server[1152]: 2026/09/23 05:14:36 INFO Signed narinfos id=38 count=15559lb # [ 120.166343] niks3-server[1152]: 2026/09/23 05:14:36 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5560lb # [ 120.176762] niks3-server[1152]: 2026/09/23 05:14:36 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5561lb # [ 120.187981] niks3-server[1152]: 2026/09/23 05:14:36 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5562worker2 # [ 120.230986] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.794Z level=INFO msg="Upload complete. (302ms)"5563lb # [ 120.191914] niks3-server[1152]: 2026/09/23 05:14:36 INFO Completed upload id=385564lb # [ 120.198961] niks3-server[1152]: 2026/09/23 05:14:36 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5565worker2 # [ 120.252906] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15566worker2 # [ 120.268474] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05567worker2 # [ 120.275150] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15568worker2 # [ 120.279420] nix-daemon[858]: reaped child process 1425, status = succeeded5569worker2 # [ 120.298789] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483705570worker2 # [ 120.315386] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483705571worker2 # [ 120.330715] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.894Z level=INFO msg="All 1 paths already cached"5572client # copying 1 paths to 'lb:50051'...5573client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5574worker2 # [ 120.341501] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483705575worker2 # [ 120.355974] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5576worker2 # [ 120.512896] nix-daemon[858]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)5577worker2 # [ 120.530598] nix-daemon[858]: reaped child process 1455, status = succeeded5578lb # [ 120.494775] niks3-server[1152]: 2026/09/23 05:14:37 INFO Received uploads request method=POST path=/api/pending_closures5579worker2 # [ 120.549664] nix-grpc-daemon[1385]: time=2026-09-23T05:14:37.113Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5580worker2 # [ 120.551356] nix-grpc-daemon[1385]: time=2026-09-23T05:14:37.113Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5581lb # [ 120.527846] niks3-server[1152]: 2026/09/23 05:14:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5582worker2 # [ 120.577860] nix-grpc-daemon[1385]: time=2026-09-23T05:14:37.141Z level=INFO msg="Uploading 1 narinfos"5583lb # [ 120.543658] niks3-server[1152]: 2026/09/23 05:14:37 INFO Signed narinfos id=39 count=15584lb # [ 120.550511] niks3-server[1152]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5585lb # [ 120.563277] niks3-server[1152]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5586worker2 # [ 120.605377] nix-grpc-daemon[1385]: time=2026-09-23T05:14:37.169Z level=INFO msg="Upload complete. (233ms)"5587lb # [ 120.571419] niks3-server[1152]: 2026/09/23 05:14:37 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5588lb # [ 120.576843] niks3-server[1152]: 2026/09/23 05:14:37 INFO Completed upload id=395589worker2 # [ 120.616538] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=0 paths=1 nar_bytes_in=6565590lb # [ 120.579066] niks3-server[1152]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5591worker2 # [ 120.631063] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483705592worker2 # [ 120.635240] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5593worker2 # [ 120.640779] nix-daemon[858]: reaped child process 1450, status = succeeded5594client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5595worker2 # [ 120.652472] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5596client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5597client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5598client # farm-b> LOG-farm-b-after-niks3-restart5599worker2 # [ 121.892260] nix-daemon[858]: reaped child process 1459, status = succeeded5600worker2 # [ 122.002629] nix-daemon[858]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5601worker2 # [ 122.021600] nix-daemon[858]: reaped child process 1478, status = succeeded5602lb # [ 121.985368] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received uploads request method=POST path=/api/pending_closures5603worker2 # [ 122.037391] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5604worker2 # [ 122.038950] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.600Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5605lb # [ 122.028703] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5606lb # [ 122.040254] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5607worker2 # [ 122.091336] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.655Z level=INFO msg="Uploading 1 narinfos"5608lb # [ 122.054359] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5609lb # [ 122.061904] niks3-server[1152]: 2026/09/23 05:14:38 INFO Signed narinfos id=40 count=15610lb # [ 122.067305] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5611lb # [ 122.079999] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5612worker2 # [ 122.128473] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.692Z level=INFO msg="Upload complete. (266ms)"5613lb # [ 122.085384] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5614lb # [ 122.094919] niks3-server[1152]: 2026/09/23 05:14:38 INFO Completed upload id=405615worker2 # [ 122.152310] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15616worker2 # [ 122.164396] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05617worker2 # [ 122.169325] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15618worker2 # [ 122.173321] nix-daemon[858]: reaped child process 1457, status = succeeded5619worker2 # [ 122.194441] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483705620worker2 # [ 122.210018] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483705621worker2 # [ 122.224916] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.788Z level=INFO msg="All 2 paths already cached"5622client # copying 1 paths to 'lb:50051'...5623client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5624worker2 # [ 122.236119] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483705625worker2 # [ 122.250072] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5626worker2 # [ 122.396803] nix-daemon[858]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)5627worker2 # [ 122.417196] nix-daemon[858]: reaped child process 1487, status = succeeded5628lb # [ 122.379353] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received uploads request method=POST path=/api/pending_closures5629worker2 # [ 122.430746] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.994Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5630worker2 # [ 122.432524] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.994Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5631lb # [ 122.408646] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5632worker2 # [ 122.462599] nix-grpc-daemon[1385]: time=2026-09-23T05:14:39.026Z level=INFO msg="Uploading 1 narinfos"5633lb # [ 122.425047] niks3-server[1152]: 2026/09/23 05:14:38 INFO Signed narinfos id=41 count=15634lb # [ 122.432269] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5635lb # [ 122.447493] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5636lb # [ 122.457376] niks3-server[1152]: 2026/09/23 05:14:39 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5637worker2 # [ 122.504732] nix-grpc-daemon[1385]: time=2026-09-23T05:14:39.068Z level=INFO msg="Upload complete. (239ms)"5638lb # [ 122.464201] niks3-server[1152]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5639lb # [ 122.471856] niks3-server[1152]: 2026/09/23 05:14:39 INFO Completed upload id=415640worker2 # [ 122.515193] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=0 paths=1 nar_bytes_in=7845641worker2 # [ 122.532450] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483705642worker2 # [ 122.541445] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5643worker2 # [ 122.544339] nix-daemon[858]: reaped child process 1482, status = succeeded5644client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5645worker2 # [ 122.556197] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5646client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5647client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5648client # farm-top> LOG-farm-top-after-niks3-restart5649worker2 # [ 123.436561] nix-daemon[858]: reaped child process 1491, status = succeeded5650worker2 # [ 123.529977] nix-daemon[858]: accepted connection from pid 1507, user nix-grpc-daemon (trusted)5651worker2 # [ 123.549829] nix-daemon[858]: reaped child process 1510, status = succeeded5652lb # [ 123.513861] niks3-server[1152]: 2026/09/23 05:14:40 INFO Received uploads request method=POST path=/api/pending_closures5653worker2 # [ 123.567133] nix-grpc-daemon[1385]: time=2026-09-23T05:14:40.130Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5654worker2 # [ 123.569264] nix-grpc-daemon[1385]: time=2026-09-23T05:14:40.130Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5655lb # [ 123.553874] niks3-server[1152]: 2026/09/23 05:14:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5656worker2 # [ 123.606226] nix-grpc-daemon[1385]: time=2026-09-23T05:14:40.170Z level=INFO msg="Uploading 1 narinfos"5657lb # [ 123.569586] niks3-server[1152]: 2026/09/23 05:14:40 INFO Signed narinfos id=42 count=15658lb # [ 123.581533] niks3-server[1152]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5659lb # [ 123.591215] niks3-server[1152]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5660lb # [ 123.603877] niks3-server[1152]: 2026/09/23 05:14:40 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5661lb # [ 123.608807] niks3-server[1152]: 2026/09/23 05:14:40 INFO Completed upload id=425662worker2 # [ 123.650667] nix-grpc-daemon[1385]: time=2026-09-23T05:14:40.214Z level=INFO msg="Upload complete. (283ms)"5663lb # [ 123.611909] niks3-server[1152]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5664lb # [ 123.615034] niks3-server[1152]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5665worker2 # [ 123.668233] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15666worker2 # [ 123.684561] nix-daemon[858]: reaped child process 1489, status = succeeded5667client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.35 seconds)5668client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5669client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.17 seconds)5670(finished: subtest: a niks3 restart does not cost the leader its role, in 7.06 seconds)5671subtest: scheduler down: the other node takes the lock and keeps it5672lb: must succeed: curl -sf localhost:9901/clusters5673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5674lb: must succeed: curl -sf localhost:9901/clusters5675lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5676lb: must succeed: curl -sf localhost:9901/clusters5677lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5678lb: must succeed: curl -sf localhost:9901/clusters5679lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5680worker2 # [ 124.125859] systemd[1]: Stopping Nix worker-protocol over gRPC...5681worker2 # [ 125.098981] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:41Z level=info event=signal_exit5682lb # [ 125.930081] niks3-server[1152]: 2026/09/23 05:14:42 INFO lead: released remote=192.168.1.4:350725683worker1 # [ 126.976062] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_restarting addr=lb:500515684worker2 # [ 126.979136] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=scheduler_restarting5685worker2 # [ 126.984720] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=scheduler_restarting addr=lb:500515686worker2 # [ 126.989245] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:483565687worker2 # [ 126.994513] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=worker_gone addr=192.168.1.3:500515688worker2 # [ 127.000266] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:483565689worker2 # [ 127.008476] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=worker_gone addr=192.168.1.4:500515690worker2 # [ 127.026966] nix-daemon[858]: reaped child process 1377, status = succeeded5691worker2 # [ 127.138958] nix-grpc-daemon[1324]: E0000 00:00:1790140483.702866 1370 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5692worker2 # [ 127.146477] nix-grpc-daemon[1324]: E0000 00:00:1790140483.710263 1370 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5693worker1 # [ 127.175709] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500515694worker1 # [ 127.181334] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5695worker2 # [ 127.195903] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5696worker2 # [ 127.251821] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5697worker2 # [ 127.256750] systemd[1]: Stopped Nix worker-protocol over gRPC.5698worker2 # [ 127.263970] systemd[1]: nix-grpc-daemon.service: Consumed 1.402s CPU time over 16.337s wall clock time, 38.9M memory peak, 47.4K incoming IP traffic, 83.4K outgoing IP traffic.5699worker2 # [ 127.276164] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5700worker2 # [ 127.282381] systemd[1]: Closed Nix worker-protocol over gRPC.5701worker2 # [ 127.285147] systemd[1]: nix-grpc-daemon.socket: Consumed 956us CPU time over 16.982s wall clock time, 256K memory peak, 38.7K incoming IP traffic, 29.4K outgoing IP traffic.5702worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5703worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5704worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5705worker1 # [ 127.381587] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500515706worker1 # [ 127.387814] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5707worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5708worker1 # [ 127.586068] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500515709worker1 # [ 127.591379] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5710worker1 # [ 127.792164] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500515711worker1 # [ 127.797168] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5712lb # [ 127.926821] niks3-server[1152]: 2026/09/23 05:14:44 INFO lead: acquired remote=192.168.1.3:432965713worker1 # [ 127.971083] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_take_over5714worker1 # [ 127.999661] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500515715worker1 # [ 128.003849] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5716worker1 # [ 128.200338] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500515717worker1 # [ 128.205379] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5718worker1 # [ 128.406203] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500515719worker1 # [ 128.411472] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5720worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5721lb # [ 128.376366] envoy[900]: conn peer=[2001:db8:1::1]:39550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9753 tx=4467 ms=60975722lb # [ 128.381291] envoy[900]: conn peer=[2001:db8:1::4]:59078 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5513 tx=665 ms=160395723worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5724worker1 # [ 128.612778] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=scheduler_connected addr=lb:500515725worker1 # [ 128.618083] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5726worker1 # [ 128.818573] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=scheduler_connected addr=lb:500515727worker1 # [ 128.827606] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:369025728worker1 # [ 128.833308] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05729worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5730worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5731client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25732client # don't know how to build these paths:5733client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5734worker1 # [ 129.774069] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:371745735worker1 # [ 129.781671] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:371745736worker1 # [ 129.808847] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05737worker1 # [ 129.815767] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15738worker1 # [ 129.854745] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371745739worker1 # [ 129.870334] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:371745740client # copying 1 paths to 'lb:50051'...5741client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5742worker1 # [ 129.887177] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:371745743worker1 # [ 129.902525] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5744worker1 # [ 129.930748] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=info event=niks3_push_started pid=12395745worker1 # [ 130.077182] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.632Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5746worker1 # [ 130.183576] nix-daemon[835]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5747worker1 # [ 130.201151] nix-daemon[835]: reaped child process 1254, status = succeeded5748lb # [ 130.164840] niks3-server[1152]: 2026/09/23 05:14:46 INFO Received uploads request method=POST path=/api/pending_closures5749worker1 # [ 130.217697] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.773Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5750worker1 # [ 130.219149] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.773Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5751lb # [ 130.203101] niks3-server[1152]: 2026/09/23 05:14:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5752worker1 # [ 130.255666] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.811Z level=INFO msg="Uploading 1 narinfos"5753lb # [ 130.216915] niks3-server[1152]: 2026/09/23 05:14:46 INFO Signed narinfos id=43 count=15754lb # [ 130.224302] niks3-server[1152]: 2026/09/23 05:14:46 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5755lb # [ 130.238799] niks3-server[1152]: 2026/09/23 05:14:46 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5756lb # [ 130.253409] niks3-server[1152]: 2026/09/23 05:14:46 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5757lb # [ 130.259820] niks3-server[1152]: 2026/09/23 05:14:46 INFO Completed upload id=435758worker1 # [ 130.301402] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.857Z level=INFO msg="Upload complete. (220ms)"5759lb # [ 130.267898] niks3-server[1152]: 2026/09/23 05:14:46 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5760worker1 # [ 130.310831] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=0 paths=1 nar_bytes_in=5205761worker1 # [ 130.327226] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371745762worker1 # [ 130.334429] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5763worker1 # [ 130.339918] nix-daemon[835]: reaped child process 1237, status = succeeded5764client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5765client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5766worker1 # [ 130.354495] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5767client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5768client # farm-a> LOG-farm-a-on-standby5769worker1 # [ 131.254959] nix-daemon[835]: reaped child process 1259, status = succeeded5770worker1 # [ 131.329235] nix-daemon[835]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5771worker1 # [ 131.345387] nix-daemon[835]: reaped child process 1278, status = succeeded5772lb # [ 131.311120] niks3-server[1152]: 2026/09/23 05:14:47 INFO Received uploads request method=POST path=/api/pending_closures5773worker1 # [ 131.364436] nix-grpc-daemon[1239]: time=2026-09-23T05:14:47.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5774worker1 # [ 131.366243] nix-grpc-daemon[1239]: time=2026-09-23T05:14:47.919Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5775lb # [ 131.340712] niks3-server[1152]: 2026/09/23 05:14:47 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5776lb # [ 131.360403] niks3-server[1152]: 2026/09/23 05:14:47 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5777worker1 # [ 131.413978] nix-grpc-daemon[1239]: time=2026-09-23T05:14:47.969Z level=INFO msg="Uploading 1 narinfos"5778lb # [ 131.371530] niks3-server[1152]: 2026/09/23 05:14:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5779lb # [ 131.377721] niks3-server[1152]: 2026/09/23 05:14:47 INFO Signed narinfos id=44 count=15780lb # [ 131.382894] niks3-server[1152]: 2026/09/23 05:14:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5781lb # [ 131.391354] niks3-server[1152]: 2026/09/23 05:14:47 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5782lb # [ 131.398712] niks3-server[1152]: 2026/09/23 05:14:47 INFO Completed upload id=445783worker1 # [ 131.441453] nix-grpc-daemon[1239]: time=2026-09-23T05:14:47.997Z level=INFO msg="Upload complete. (277ms)"5784lb # [ 131.403654] niks3-server[1152]: 2026/09/23 05:14:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5785worker1 # [ 131.465105] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15786worker1 # [ 131.478710] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05787worker1 # [ 131.482434] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15788worker1 # [ 131.486274] nix-daemon[835]: reaped child process 1257, status = succeeded5789worker1 # [ 131.507370] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371745790worker1 # [ 131.522359] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:371745791worker1 # [ 131.536045] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.092Z level=INFO msg="All 1 paths already cached"5792client # copying 1 paths to 'lb:50051'...5793client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5794worker1 # [ 131.546824] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:371745795worker1 # [ 131.560475] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5796worker1 # [ 131.711396] nix-daemon[835]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5797worker1 # [ 131.728273] nix-daemon[835]: reaped child process 1287, status = succeeded5798lb # [ 131.692480] niks3-server[1152]: 2026/09/23 05:14:48 INFO Received uploads request method=POST path=/api/pending_closures5799worker1 # [ 131.744091] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.299Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5800worker1 # [ 131.745695] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.299Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5801lb # [ 131.720697] niks3-server[1152]: 2026/09/23 05:14:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5802worker1 # [ 131.771056] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.326Z level=INFO msg="Uploading 1 narinfos"5803lb # [ 131.734332] niks3-server[1152]: 2026/09/23 05:14:48 INFO Signed narinfos id=45 count=15804lb # [ 131.743036] niks3-server[1152]: 2026/09/23 05:14:48 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5805lb # [ 131.755721] niks3-server[1152]: 2026/09/23 05:14:48 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5806worker1 # [ 131.806962] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.363Z level=INFO msg="Upload complete. (230ms)"5807lb # [ 131.767789] niks3-server[1152]: 2026/09/23 05:14:48 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5808lb # [ 131.774477] niks3-server[1152]: 2026/09/23 05:14:48 INFO Completed upload id=455809worker1 # [ 131.816265] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=0 paths=1 nar_bytes_in=6405810lb # [ 131.778691] niks3-server[1152]: 2026/09/23 05:14:48 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5811worker1 # [ 131.832250] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371745812worker1 # [ 131.838619] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5813worker1 # [ 131.843643] nix-daemon[835]: reaped child process 1282, status = succeeded5814client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5815worker1 # [ 131.854615] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5816client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5817client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5818client # farm-b> LOG-farm-b-on-standby5819worker1 # [ 132.922100] nix-daemon[835]: reaped child process 1291, status = succeeded5820worker1 # [ 133.039260] nix-daemon[835]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5821worker1 # [ 133.057183] nix-daemon[835]: reaped child process 1310, status = succeeded5822lb # [ 133.021476] niks3-server[1152]: 2026/09/23 05:14:49 INFO Received uploads request method=POST path=/api/pending_closures5823worker1 # [ 133.075956] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.631Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5824worker1 # [ 133.077965] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.631Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5825lb # [ 133.062754] niks3-server[1152]: 2026/09/23 05:14:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5826worker1 # [ 133.114795] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.670Z level=INFO msg="Uploading 1 narinfos"5827lb # [ 133.079785] niks3-server[1152]: 2026/09/23 05:14:49 INFO Signed narinfos id=46 count=15828lb # [ 133.088815] niks3-server[1152]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5829lb # [ 133.097670] niks3-server[1152]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5830lb # [ 133.103129] niks3-server[1152]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5831lb # [ 133.116413] niks3-server[1152]: 2026/09/23 05:14:49 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5832worker1 # [ 133.158819] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.715Z level=INFO msg="Upload complete. (292ms)"5833lb # [ 133.120326] niks3-server[1152]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5834lb # [ 133.125932] niks3-server[1152]: 2026/09/23 05:14:49 INFO Completed upload id=465835worker1 # [ 133.176279] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15836worker1 # [ 133.195860] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05837worker1 # [ 133.201009] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15838worker1 # [ 133.205658] nix-daemon[835]: reaped child process 1289, status = succeeded5839worker1 # [ 133.235570] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371745840worker1 # [ 133.252475] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:371745841worker1 # [ 133.267431] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.823Z level=INFO msg="All 2 paths already cached"5842client # copying 1 paths to 'lb:50051'...5843client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5844worker1 # [ 133.278987] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:371745845worker1 # [ 133.292830] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5846worker1 # [ 133.454390] nix-daemon[835]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5847worker1 # [ 133.474205] nix-daemon[835]: reaped child process 1319, status = succeeded5848lb # [ 133.434689] niks3-server[1152]: 2026/09/23 05:14:50 INFO Received uploads request method=POST path=/api/pending_closures5849worker1 # [ 133.488064] nix-grpc-daemon[1239]: time=2026-09-23T05:14:50.044Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5850worker1 # [ 133.489674] nix-grpc-daemon[1239]: time=2026-09-23T05:14:50.044Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5851lb # [ 133.471518] niks3-server[1152]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5852worker1 # [ 133.522854] nix-grpc-daemon[1239]: time=2026-09-23T05:14:50.078Z level=INFO msg="Uploading 1 narinfos"5853lb # [ 133.483375] niks3-server[1152]: 2026/09/23 05:14:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5854lb # [ 133.495260] niks3-server[1152]: 2026/09/23 05:14:50 INFO Signed narinfos id=47 count=15855lb # [ 133.501264] niks3-server[1152]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5856worker1 # [ 133.555755] nix-grpc-daemon[1239]: time=2026-09-23T05:14:50.111Z level=INFO msg="Upload complete. (245ms)"5857lb # [ 133.514057] niks3-server[1152]: 2026/09/23 05:14:50 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5858lb # [ 133.518093] niks3-server[1152]: 2026/09/23 05:14:50 INFO Completed upload id=475859lb # [ 133.521894] niks3-server[1152]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5860worker1 # [ 133.568332] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=0 paths=1 nar_bytes_in=7685861worker1 # [ 133.585237] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371745862worker1 # [ 133.592730] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5863worker1 # [ 133.598366] nix-daemon[835]: reaped child process 1314, status = succeeded5864client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5865worker1 # [ 133.610361] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5866client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5867client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5868client # farm-top> LOG-farm-top-on-standby5869worker1 # [ 134.870758] nix-daemon[835]: reaped child process 1323, status = succeeded5870worker1 # [ 134.986934] nix-daemon[835]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5871worker1 # [ 135.009945] nix-daemon[835]: reaped child process 1342, status = succeeded5872lb # [ 134.967281] niks3-server[1152]: 2026/09/23 05:14:51 INFO Received uploads request method=POST path=/api/pending_closures5873worker1 # [ 135.020419] nix-grpc-daemon[1239]: time=2026-09-23T05:14:51.575Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5874worker1 # [ 135.021928] nix-grpc-daemon[1239]: time=2026-09-23T05:14:51.575Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5875lb # [ 135.010946] niks3-server[1152]: 2026/09/23 05:14:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5876worker1 # [ 135.065527] nix-grpc-daemon[1239]: time=2026-09-23T05:14:51.621Z level=INFO msg="Uploading 1 narinfos"5877lb # [ 135.023766] niks3-server[1152]: 2026/09/23 05:14:51 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5878lb # [ 135.033563] niks3-server[1152]: 2026/09/23 05:14:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5879lb # [ 135.041688] niks3-server[1152]: 2026/09/23 05:14:51 INFO Signed narinfos id=48 count=15880lb # [ 135.049054] niks3-server[1152]: 2026/09/23 05:14:51 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5881worker1 # [ 135.097492] nix-grpc-daemon[1239]: time=2026-09-23T05:14:51.653Z level=INFO msg="Upload complete. (273ms)"5882lb # [ 135.057032] niks3-server[1152]: 2026/09/23 05:14:51 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5883lb # [ 135.060708] niks3-server[1152]: 2026/09/23 05:14:51 INFO Completed upload id=485884lb # [ 135.065823] niks3-server[1152]: 2026/09/23 05:14:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5885worker1 # [ 135.113214] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15886worker1 # [ 135.129521] nix-daemon[835]: reaped child process 1321, status = succeeded5887client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.61 seconds)5888client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5889client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5890worker2 # [ 135.414777] systemd[1]: Listening on Nix worker-protocol over gRPC.5891worker2 # [ 135.423594] systemd[1]: Starting scheduler-token.service...5892worker2 # [ 135.500295] systemd[1]: scheduler-token.service: Deactivated successfully.5893worker2 # [ 135.508239] systemd[1]: Finished scheduler-token.service.5894worker2 # [ 135.509915] systemd[1]: scheduler-token.service: Consumed 44ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5895worker2 # [ 135.526389] systemd[1]: Starting Nix worker-protocol over gRPC...5896worker2 # [ 135.643716] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5897worker2 # [ 135.646211] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5898worker2 # [ 135.650179] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15899worker2 # [ 135.656069] nix-grpc-daemon[1531]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5900worker2 # [ 135.657609] nix-grpc-daemon[1531]: E0000 00:00:1790140492.218880 1531 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5901lb: must succeed: curl -sf localhost:9901/clusters5902worker2 # [ 135.702179] nix-grpc-daemon[1531]: E0000 00:00:1790140492.225229 1531 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5903worker2 # [ 135.705233] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5904worker2 # [ 135.707130] nix-grpc-daemon[1531]: E0000 00:00:1790140492.234315 1572 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5905worker2 # [ 135.709713] nix-grpc-daemon[1531]: E0000 00:00:1790140492.250866 1572 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5906worker2 # [ 135.712091] nix-grpc-daemon[1531]: E0000 00:00:1790140492.251009 1572 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5907worker2 # [ 135.717630] nix-grpc-daemon[1531]: E0000 00:00:1790140492.265445 1572 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5908worker2 # [ 135.722237] systemd[1]: Started Nix worker-protocol over gRPC.5909worker2 # [ 135.723161] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=scheduler_connected addr=lb:500515910worker1 # [ 135.732209] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=info event=oidc_jwks provider=mock keys=15911worker1 # [ 135.736813] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:369025912worker1 # [ 135.742078] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05913lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5914worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5915worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5916lb: must succeed: curl -sf localhost:9901/clusters5917lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5918lb: must succeed: curl -sf localhost:9901/clusters5919lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5920worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5921worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5922(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.10 seconds)5923subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5924client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5925client # [ 136.050115] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5926client # Running as unit: rs.service; invocation ID: f1a868cf617c401ca5c0cac25df23f685927client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5928worker1 # [ 136.242629] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:371745929client # [ 136.250809] nix[1400]: don't know how to build these paths:5930client # [ 136.254113] nix[1400]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5931worker1 # [ 136.262306] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05932worker1 # [ 136.269608] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15933worker1 # [ 136.307089] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371745934worker1 # [ 136.322445] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:371745935client # [ 136.336343] nix[1400]: copying 1 paths to 'lb:50051'...5936worker1 # [ 136.337288] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:371745937client # [ 136.340184] nix[1400]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5938worker1 # [ 136.352708] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5939worker1 # [ 136.511807] nix-daemon[835]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)5940worker1 # [ 136.531407] nix-daemon[835]: reaped child process 1358, status = succeeded5941lb # [ 136.498372] niks3-server[1152]: 2026/09/23 05:14:53 INFO Received uploads request method=POST path=/api/pending_closures5942worker1 # [ 136.550548] nix-grpc-daemon[1239]: time=2026-09-23T05:14:53.106Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5943worker1 # [ 136.552289] nix-grpc-daemon[1239]: time=2026-09-23T05:14:53.106Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5944lb # [ 136.527537] niks3-server[1152]: 2026/09/23 05:14:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5945worker1 # [ 136.578640] nix-grpc-daemon[1239]: time=2026-09-23T05:14:53.134Z level=INFO msg="Uploading 1 narinfos"5946lb # [ 136.542748] niks3-server[1152]: 2026/09/23 05:14:53 INFO Signed narinfos id=49 count=15947lb # [ 136.549235] niks3-server[1152]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5948lb # [ 136.560252] niks3-server[1152]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5949worker1 # [ 136.602613] nix-grpc-daemon[1239]: time=2026-09-23T05:14:53.158Z level=INFO msg="Upload complete. (233ms)"5950lb # [ 136.563236] niks3-server[1152]: 2026/09/23 05:14:53 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5951lb # [ 136.569051] niks3-server[1152]: 2026/09/23 05:14:53 INFO Completed upload id=495952worker1 # [ 136.612304] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=0 paths=1 nar_bytes_in=4165953lb # [ 136.574590] niks3-server[1152]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5954worker1 # [ 136.624767] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:371745955worker1 # [ 136.634369] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5956worker1 # [ 136.644347] nix-daemon[835]: reaped child process 1353, status = succeeded5957client # [ 136.649581] nix[1400]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5958worker1 # [ 136.652647] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5959lb # [ 138.375857] envoy[900]: conn peer=[2001:db8:1::1]:42962 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9619 tx=4412 ms=53545960lb: must succeed: curl -sf localhost:9901/clusters5961lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5962lb: must succeed: curl -sf localhost:9901/clusters5963lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5964lb: must succeed: curl -sf localhost:9901/clusters5965lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5966lb: must succeed: curl -sf localhost:9901/clusters5967lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5968worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5969worker2 # [ 138.846360] systemd-journald[270]: Received client request to rotate journal, rotating.5970worker2 # [ 138.864909] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/7c021b444f874723a0e9fb1639a0b9ab.5971worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5972worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5973worker1 # [ 138.928998] systemd[1]: Stopping Nix worker-protocol over gRPC...5974worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5975client: must succeed: sleep 5.05976worker1 # [ 139.687092] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:56Z level=info event=draining5977client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5978worker1 # [ 144.286967] nix-daemon[835]: reaped child process 1362, status = succeeded5979worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5980worker1 # [ 144.413518] nix-daemon[835]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)5981worker1 # [ 144.435302] nix-daemon[835]: reaped child process 1406, status = succeeded5982lb # [ 144.398608] niks3-server[1152]: 2026/09/23 05:15:00 INFO Received uploads request method=POST path=/api/pending_closures5983worker1 # [ 144.456456] nix-grpc-daemon[1239]: time=2026-09-23T05:15:01.010Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5984worker1 # [ 144.458028] nix-grpc-daemon[1239]: time=2026-09-23T05:15:01.010Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5985lb # [ 144.471323] niks3-server[1152]: 2026/09/23 05:15:01 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5986lb # [ 144.492142] niks3-server[1152]: 2026/09/23 05:15:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5987lb # [ 144.502881] niks3-server[1152]: 2026/09/23 05:15:01 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5988lb # [ 144.519561] niks3-server[1152]: 2026/09/23 05:15:01 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5989lb # [ 144.529784] niks3-server[1152]: 2026/09/23 05:15:01 INFO Signed narinfos id=50 count=15990worker1 # [ 144.576116] nix-grpc-daemon[1239]: time=2026-09-23T05:15:01.131Z level=INFO msg="Uploading 1 narinfos"5991lb # [ 144.544824] niks3-server[1152]: 2026/09/23 05:15:01 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5992lb # [ 144.560379] niks3-server[1152]: 2026/09/23 05:15:01 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5993lb # [ 144.571109] niks3-server[1152]: 2026/09/23 05:15:01 INFO Completed upload id=505994worker1 # [ 144.614123] nix-grpc-daemon[1239]: time=2026-09-23T05:15:01.170Z level=INFO msg="Upload complete. (344ms)"5995worker1 # [ 144.633838] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15996worker1 # [ 144.656445] nix-daemon[835]: reaped child process 1360, status = succeeded5997client # [ 144.670967] systemd[1]: rs.service: Deactivated successfully.5998client # [ 144.676396] systemd[1]: rs.service: Consumed 194ms CPU time over 8.625s wall clock time, 39.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.5999worker1 # [ 144.688906] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:01Z level=info event=signal_exit6000lb # [ 144.932655] niks3-server[1152]: 2026/09/23 05:15:01 INFO lead: released remote=192.168.1.3:432966001lb # [ 145.631815] niks3-server[1152]: 2026/09/23 05:15:02 INFO lead: acquired remote=192.168.1.4:449126002worker2 # [ 145.674597] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_take_over6003worker2 # [ 145.975188] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_restarting addr=lb:500516004worker1 # [ 145.982220] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=scheduler_restarting6005worker1 # [ 145.992383] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=scheduler_restarting addr=lb:500516006worker1 # [ 145.997571] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:369026007worker1 # [ 146.003551] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=worker_gone addr=192.168.1.4:500516008worker1 # [ 146.009512] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:369026009worker1 # [ 146.014572] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=worker_gone addr=192.168.1.3:500516010worker1 # [ 146.044456] nix-daemon[835]: reaped child process 1212, status = succeeded6011worker2 # [ 146.179139] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_connected addr=lb:500516012worker2 # [ 146.184484] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6013worker1 # [ 146.207655] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6014worker2 # [ 146.386005] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_connected addr=lb:500516015worker2 # [ 146.408165] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=oidc_jwks provider=mock keys=16016worker2 # [ 146.413871] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:435786017worker2 # [ 146.419540] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06018worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.23 seconds)6019client: waiting for success: ! systemctl is-active rs6020client: (finished: waiting for success: ! systemctl is-active rs, in 0.08 seconds)6021client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6022client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)6023client: must fail: journalctl -u rs | grep -q 'reconnecting'6024client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6025lb: must succeed: curl -sf localhost:9901/clusters6026lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)6027worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6028worker1 # [ 146.844341] nix-grpc-daemon[1159]: E0000 00:00:1790140503.399790 1205 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6029worker1 # [ 146.851280] nix-grpc-daemon[1159]: E0000 00:00:1790140503.400272 1205 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6030worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6031worker1 # [ 147.880389] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6032worker1 # [ 147.886773] systemd[1]: Stopped Nix worker-protocol over gRPC.6033worker1 # [ 147.890410] systemd[1]: nix-grpc-daemon.service: Consumed 1.633s CPU time over 32.381s wall clock time, 39.2M memory peak, 48.2K incoming IP traffic, 99.6K outgoing IP traffic.6034lb: must succeed: curl -sf localhost:9901/clusters6035worker1 # [ 147.914666] systemd[1]: Starting Nix worker-protocol over gRPC...6036lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.13 seconds)6037worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6038worker1 # [ 148.083525] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6039worker1 # [ 148.089786] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6040worker1 # [ 148.112135] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16041worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6042worker1 # [ 148.136326] nix-grpc-daemon[1413]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6043worker1 # [ 148.137822] nix-grpc-daemon[1413]: E0000 00:00:1790140504.651225 1413 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6044worker1 # [ 148.149001] nix-grpc-daemon[1413]: E0000 00:00:1790140504.655207 1413 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6045worker1 # [ 148.152911] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6046worker1 # [ 148.154710] nix-grpc-daemon[1413]: E0000 00:00:1790140504.663271 1453 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6047worker1 # [ 148.157138] nix-grpc-daemon[1413]: E0000 00:00:1790140504.682646 1453 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6048worker1 # [ 148.163438] nix-grpc-daemon[1413]: E0000 00:00:1790140504.682845 1453 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6049worker1 # [ 148.168127] systemd[1]: Started Nix worker-protocol over gRPC.6050worker1 # [ 148.169059] nix-grpc-daemon[1413]: E0000 00:00:1790140504.703955 1453 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6051worker1 # [ 148.171558] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=scheduler_connected addr=lb:500516052worker2 # [ 148.175670] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:04Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:435786053worker2 # [ 148.180985] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06054lb # [ 148.374533] envoy[900]: conn peer=[2001:db8:1::1]:39886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1540 ms=83996055lb # [ 148.380490] envoy[900]: conn peer=[2001:db8:1::3]:50484 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2608 tx=921 ms=304616056lb: must succeed: curl -sf localhost:9901/clusters6057lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)6058worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6059worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6060(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.33 seconds)6061subtest: scheduler restart mid-build: build finishes, no double build6062client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid6063client # [ 149.397500] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.6064client # Running as unit: mid.service; invocation ID: 77846b5a7d4949f9b6817840c77f1b8d6065client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.12 seconds)6066worker1 # [ 149.634237] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:381166067worker1 # [ 149.648424] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6068client # [ 149.670751] nix[1435]: don't know how to build these paths:6069client # [ 149.673255] nix[1435]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6070worker2 # [ 149.686669] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:06Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06071worker2 # [ 149.692250] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:06Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16072worker2 # [ 149.750150] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435946073worker2 # [ 149.767245] nix-daemon[858]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)6074worker2 # [ 149.794291] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)6075worker2 # [ 149.851441] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 182 ms (attempt 2/5)6076worker2 # [ 150.035572] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 3/5)6077worker2 # [ 150.057328] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 515 ms (attempt 4/5)6078worker2 # [ 150.576232] nix-grpc-daemon[1531]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6079worker2 # [ 150.582824] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6080worker2 # [ 150.603187] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435946081client # [ 150.620166] nix[1435]: copying 1 paths to 'lb:50051'...6082worker2 # [ 150.619122] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435946083client # [ 150.623658] nix[1435]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...6084worker2 # [ 150.640583] nix-daemon[858]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)6085worker2 # [ 150.674967] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=info event=niks3_push_started pid=16606086worker2 # [ 150.845259] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.408Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6087worker2 # [ 150.953400] nix-daemon[858]: accepted connection from pid 1674, user nix-grpc-daemon (trusted)6088worker2 # [ 150.972856] nix-daemon[858]: reaped child process 1677, status = succeeded6089lb # [ 150.935637] niks3-server[1152]: 2026/09/23 05:15:07 INFO Received uploads request method=POST path=/api/pending_closures6090worker2 # [ 150.989790] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.552Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6091worker2 # [ 150.991277] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.552Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"6092lb # [ 150.985549] niks3-server[1152]: 2026/09/23 05:15:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6093worker2 # [ 151.040912] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.603Z level=INFO msg="Uploading 1 narinfos"6094lb # [ 151.000288] niks3-server[1152]: 2026/09/23 05:15:07 INFO Signed narinfos id=51 count=16095lb # [ 151.002807] niks3-server[1152]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls6096lb # [ 151.013596] niks3-server[1152]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst6097lb # [ 151.023906] niks3-server[1152]: 2026/09/23 05:15:07 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6098worker2 # [ 151.071789] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.635Z level=INFO msg="Upload complete. (222ms)"6099lb # [ 151.032247] niks3-server[1152]: 2026/09/23 05:15:07 INFO Completed upload id=516100lb # [ 151.035870] niks3-server[1152]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo6101worker2 # [ 151.080905] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43594 duration_s=0 paths=1 nar_bytes_in=4246102worker2 # [ 151.097719] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435946103worker2 # [ 151.103939] nix-daemon[858]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)6104worker2 # [ 151.109331] nix-daemon[858]: reaped child process 1655, status = succeeded6105worker2 # [ 151.121532] nix-daemon[858]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)6106client # [ 151.120311] nix[1435]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6107lb: must succeed: curl -sf localhost:9901/clusters6108lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)6109lb: must succeed: curl -sf localhost:9901/clusters6110lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6111worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6112worker2 # [ 152.291766] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1531 (nix-grpc-daemon) on client request.6113worker2 # [ 152.293946] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6114lb # [ 152.254607] niks3-server[1152]: 2026/09/23 05:15:08 INFO lead: released remote=192.168.1.4:449126115client # [ 152.304568] nix[1435]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6116worker2 # [ 152.308891] systemd[1]: Starting scheduler-token.service...6117worker1 # [ 152.304708] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:08Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6118client # [ 152.309718] nix[1435]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6119worker2 # [ 152.317841] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6120worker2 # [ 152.319315] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6121worker2 # [ 152.326208] systemd[1]: nix-grpc-daemon.service: Consumed 541ms CPU time over 16.795s wall clock time, 34.6M memory peak, 15.8K incoming IP traffic, 28.5K outgoing IP traffic.6122worker2 # [ 152.367903] nix-daemon[858]: reaped child process 1679, status = succeeded6123worker2 # [ 152.371344] nix-daemon[858]: reaped child process 1649, status = succeeded6124worker2 # [ 152.380531] nix-daemon[1681]: unexpected Nix daemon error: error: interrupted by the user6125worker2 # [ 152.393166] nix-daemon[858]: reaped child process 1681, status = failed with exit code 16126worker2 # [ 152.410614] systemd[1]: scheduler-token.service: Deactivated successfully.6127worker2 # [ 152.411572] systemd[1]: Finished scheduler-token.service.6128worker2 # [ 152.414176] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 98ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6129worker2 # [ 152.419813] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6130worker2 # [ 152.421427] systemd[1]: Starting scheduler-token.service...6131worker2 # [ 152.460905] systemd[1]: scheduler-token.service: Deactivated successfully.6132worker2 # [ 152.464158] systemd[1]: Finished scheduler-token.service.6133worker2 # [ 152.464983] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 48ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6134worker2 # [ 152.468807] systemd[1]: Starting Nix worker-protocol over gRPC...6135worker1 # [ 152.504869] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:09Z level=info event=scheduler_connected addr=lb:500516136worker2 # [ 152.570148] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6137worker2 # [ 152.572393] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6138worker2 # [ 152.576228] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16139worker2 # [ 152.582626] nix-grpc-daemon[1720]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6140worker2 # [ 152.600042] nix-grpc-daemon[1720]: E0000 00:00:1790140509.144545 1720 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6141worker2 # [ 152.602448] nix-grpc-daemon[1720]: E0000 00:00:1790140509.146570 1720 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6142lb # [ 152.561075] niks3-server[1152]: 2026/09/23 05:15:09 INFO lead: acquired remote=192.168.1.4:598806143worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.38 seconds)6144client: must succeed: sleep 5.06145worker2 # [ 152.638976] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6146worker2 # [ 152.641192] nix-grpc-daemon[1720]: E0000 00:00:1790140509.162550 1763 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6147worker2 # [ 152.643684] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=scheduler_take_over6148worker2 # [ 152.644887] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:408666149worker2 # [ 152.649263] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06150worker2 # [ 152.653733] nix-grpc-daemon[1720]: E0000 00:00:1790140509.187564 1763 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6151worker2 # [ 152.656775] nix-grpc-daemon[1720]: E0000 00:00:1790140509.187722 1763 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6152worker2 # [ 152.659610] nix-grpc-daemon[1720]: E0000 00:00:1790140509.202340 1763 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6153worker2 # [ 152.663961] systemd[1]: Started Nix worker-protocol over gRPC.6154worker2 # [ 152.664908] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=scheduler_connected addr=lb:500516155worker2 # [ 152.666146] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=oidc_jwks provider=mock keys=16156worker2 # [ 152.667380] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:408666157worker2 # [ 152.669136] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06158worker2 # [ 152.812251] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06159worker2 # [ 152.818149] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16160worker1 # [ 152.863034] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381166161worker1 # [ 152.880228] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5)6162worker1 # [ 152.967425] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 2/5)6163worker1 # [ 153.054334] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5)6164worker1 # [ 153.173340] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 253 ms (attempt 4/5)6165worker1 # [ 153.429827] nix-grpc-daemon[1413]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6166worker1 # [ 153.437676] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6167worker1 # [ 153.452910] nix-grpc-daemon[1413]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...6168worker1 # [ 153.478313] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6169client # [ 153.500722] nix[1435]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6170worker1 # [ 153.504257] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6171client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6172worker1 # [ 157.924595] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:14Z level=info event=niks3_push_started pid=15166173client: waiting for success: ! systemctl is-active mid6174worker1 # [ 157.960532] nix-daemon[835]: reaped child process 1491, status = succeeded6175worker1 # [ 158.059406] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.615Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6176worker1 # [ 158.175127] nix-daemon[835]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)6177worker1 # [ 158.190505] nix-daemon[835]: reaped child process 1532, status = succeeded6178lb # [ 158.157558] niks3-server[1152]: 2026/09/23 05:15:14 INFO Received uploads request method=POST path=/api/pending_closures6179worker1 # [ 158.212064] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.767Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6180worker1 # [ 158.213528] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.767Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6181lb # [ 158.219067] niks3-server[1152]: 2026/09/23 05:15:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6182lb # [ 158.232082] niks3-server[1152]: 2026/09/23 05:15:14 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6183lb # [ 158.244879] niks3-server[1152]: 2026/09/23 05:15:14 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6184worker1 # [ 158.303201] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.857Z level=INFO msg="Uploading 1 narinfos"6185lb # [ 158.258908] niks3-server[1152]: 2026/09/23 05:15:14 INFO Signed narinfos id=52 count=16186lb # [ 158.268909] niks3-server[1152]: 2026/09/23 05:15:14 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6187lb # [ 158.276804] niks3-server[1152]: 2026/09/23 05:15:14 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6188lb # [ 158.283040] niks3-server[1152]: 2026/09/23 05:15:14 INFO Completed upload id=526189worker1 # [ 158.325924] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.882Z level=INFO msg="Upload complete. (262ms)"6190lb # [ 158.286795] niks3-server[1152]: 2026/09/23 05:15:14 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6191worker1 # [ 158.340388] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38116 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16192worker1 # [ 158.357398] nix-daemon[835]: reaped child process 1489, status = succeeded6193client # [ 158.374024] systemd[1]: mid.service: Deactivated successfully.6194client # [ 158.379030] systemd[1]: mid.service: Consumed 236ms CPU time over 8.979s wall clock time, 43.5M memory peak, 8.7K incoming IP traffic, 10.3K outgoing IP traffic.6195lb # [ 158.375814] envoy[900]: conn peer=[2001:db8:1::4]:43202 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2709 tx=393 ms=165966196lb # [ 158.381233] envoy[900]: conn peer=[2001:db8:1::1]:39328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3927 tx=1942 ms=87276197client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6198client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6199client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)6200(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.80 seconds)6201subtest: interrupting the client stops the build on the worker6202client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6203client # [ 159.144529] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6204client # Running as unit: intr.service; invocation ID: d09c4e7be0b340bb930643bd0ea748106205client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6206worker2 # [ 159.325937] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:408826207worker2 # [ 159.338743] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6208client # [ 159.354954] nix[1471]: don't know how to build these paths:6209client # [ 159.356339] nix[1471]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6210worker2 # [ 159.368592] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:15Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06211worker2 # [ 159.375312] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:15Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16212worker2 # [ 159.422872] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826213worker2 # [ 159.438876] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)6214worker2 # [ 159.463021] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 2/5)6215worker2 # [ 159.531812] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 3/5)6216worker2 # [ 159.921758] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 278 ms (attempt 4/5)6217worker2 # [ 160.229851] nix-grpc-daemon[1720]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6218worker2 # [ 160.236233] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6219worker2 # [ 160.253530] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408826220client # [ 160.268404] nix[1471]: copying 1 paths to 'lb:50051'...6221worker2 # [ 160.268748] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408826222client # [ 160.268700] nix[1471]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6223worker2 # [ 160.285169] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6224worker2 # [ 160.317165] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:16Z level=info event=niks3_push_started pid=17976225worker2 # [ 160.478632] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.040Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6226worker2 # [ 160.586014] nix-daemon[858]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)6227worker2 # [ 160.605976] nix-daemon[858]: reaped child process 1817, status = succeeded6228lb # [ 160.563145] niks3-server[1152]: 2026/09/23 05:15:17 INFO Received uploads request method=POST path=/api/pending_closures6229worker2 # [ 160.618310] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.181Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6230worker2 # [ 160.622087] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.181Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6231lb # [ 160.606979] niks3-server[1152]: 2026/09/23 05:15:17 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6232worker2 # [ 160.660063] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.223Z level=INFO msg="Uploading 1 narinfos"6233lb # [ 160.623161] niks3-server[1152]: 2026/09/23 05:15:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6234lb # [ 160.631934] niks3-server[1152]: 2026/09/23 05:15:17 INFO Signed narinfos id=53 count=16235lb # [ 160.640542] niks3-server[1152]: 2026/09/23 05:15:17 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6236lb # [ 160.643918] niks3-server[1152]: 2026/09/23 05:15:17 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6237worker2 # [ 160.692983] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.256Z level=INFO msg="Upload complete. (215ms)"6238lb # [ 160.652119] niks3-server[1152]: 2026/09/23 05:15:17 INFO Completed upload id=536239lb # [ 160.654623] niks3-server[1152]: 2026/09/23 05:15:17 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6240worker2 # [ 160.700846] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=4246241worker2 # [ 160.714038] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826242worker2 # [ 160.723028] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6243worker2 # [ 160.725515] nix-daemon[858]: reaped child process 1795, status = succeeded6244worker2 # [ 160.737711] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6245client # [ 160.736595] nix[1471]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6246client: must succeed: systemctl kill -s INT intr6247worker2 # [ 162.796463] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:19Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6248client # [ 162.795036] systemd[1]: intr.service: Sent signal SIGINT to main process 1471 (nix) on client request.6249client # [ 162.801170] nix[1471]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6250client # [ 162.813927] nix[1471]: error: interrupted by the user6251client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6252client # [ 162.839944] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6253client # [ 162.845025] systemd[1]: intr.service: Failed with result 'exit-code'.6254client # [ 162.847326] systemd[1]: intr.service: Consumed 171ms CPU time over 3.698s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6255worker2 # [ 162.963364] nix-daemon[1821]: unexpected Nix daemon error: error: interrupted by the user6256worker2 # [ 162.968687] nix-daemon[858]: reaped child process 1819, status = succeeded6257worker2 # [ 162.973786] nix-daemon[858]: reaped child process 1821, status = failed with exit code 16258(finished: subtest: interrupting the client stops the build on the worker, in 4.99 seconds)6259subtest: a deploy mid-build drains: the build finishes, the new generation takes over6260client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6261client # [ 164.158618] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6262client # Running as unit: sw.service; invocation ID: 6f7747b2456d46d9bfe51f4489d710df6263client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)6264client # [ 164.367447] nix[1492]: don't know how to build these paths:6265client # [ 164.368548] nix[1492]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6266worker1 # [ 164.366886] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:381166267worker2 # [ 164.380757] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:20Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06268worker2 # [ 164.386140] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:20Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16269worker1 # [ 164.426940] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381166270worker1 # [ 164.443342] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381166271client # [ 164.458325] nix[1492]: copying 1 paths to 'lb:50051'...6272client # [ 164.463823] nix[1492]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6273worker1 # [ 164.463135] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381166274worker1 # [ 164.478157] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6275worker1 # [ 164.644367] nix-daemon[835]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6276worker1 # [ 164.665229] nix-daemon[835]: reaped child process 1567, status = succeeded6277lb # [ 164.627974] niks3-server[1152]: 2026/09/23 05:15:21 INFO Received uploads request method=POST path=/api/pending_closures6278worker1 # [ 164.683294] nix-grpc-daemon[1516]: time=2026-09-23T05:15:21.238Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6279worker1 # [ 164.685220] nix-grpc-daemon[1516]: time=2026-09-23T05:15:21.238Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6280worker1 # [ 164.703269] nix-grpc-daemon[1516]: time=2026-09-23T05:15:21.258Z level=INFO msg="Uploading 1 narinfos"6281lb # [ 164.660428] niks3-server[1152]: 2026/09/23 05:15:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6282lb # [ 164.675931] niks3-server[1152]: 2026/09/23 05:15:21 INFO Signed narinfos id=54 count=16283lb # [ 164.690363] niks3-server[1152]: 2026/09/23 05:15:21 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6284lb # [ 164.704610] niks3-server[1152]: 2026/09/23 05:15:21 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6285lb # [ 164.717587] niks3-server[1152]: 2026/09/23 05:15:21 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6286worker1 # [ 164.756474] nix-grpc-daemon[1516]: time=2026-09-23T05:15:21.312Z level=INFO msg="Upload complete. (261ms)"6287lb # [ 164.726161] niks3-server[1152]: 2026/09/23 05:15:21 INFO Completed upload id=546288worker1 # [ 164.769199] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38116 duration_s=0 paths=1 nar_bytes_in=4166289lb # [ 164.729730] niks3-server[1152]: 2026/09/23 05:15:21 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6290worker1 # [ 164.784762] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381166291worker1 # [ 164.794488] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6292worker1 # [ 164.800777] nix-daemon[835]: reaped child process 1562, status = succeeded6293worker1 # [ 164.814668] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6294client # [ 164.814081] nix[1492]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6295worker1: must succeed: systemctl reload nix-grpc-daemon.service6296worker1 # [ 166.780221] systemd[1]: Reloading Nix worker-protocol over gRPC...6297worker1 # [ 166.816983] systemd[1]: Reloaded Nix worker-protocol over gRPC.6298worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6299worker1: must succeed: pgrep -f 'read -t [9]0 x'6300worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6301client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26302client # don't know how to build these paths:6303client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6304worker1 # [ 167.097156] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381166305worker2 # [ 167.094423] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:408826306worker1 # [ 167.111117] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:23Z level=info event=draining6307worker2 # [ 167.119912] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06308worker2 # [ 167.126552] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16309worker2 # [ 167.171055] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826310worker2 # [ 167.186183] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408826311client # copying 1 paths to 'lb:50051'...6312worker2 # [ 167.202156] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408826313client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6314worker2 # [ 167.225650] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6315worker2 # [ 167.383826] nix-daemon[858]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6316worker2 # [ 167.402369] nix-daemon[858]: reaped child process 1879, status = succeeded6317lb # [ 167.367383] niks3-server[1152]: 2026/09/23 05:15:23 INFO Received uploads request method=POST path=/api/pending_closures6318worker2 # [ 167.419436] nix-grpc-daemon[1797]: time=2026-09-23T05:15:23.982Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6319worker2 # [ 167.421234] nix-grpc-daemon[1797]: time=2026-09-23T05:15:23.982Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6320lb # [ 167.396627] niks3-server[1152]: 2026/09/23 05:15:23 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6321worker2 # [ 167.452265] nix-grpc-daemon[1797]: time=2026-09-23T05:15:24.016Z level=INFO msg="Uploading 1 narinfos"6322lb # [ 167.409981] niks3-server[1152]: 2026/09/23 05:15:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6323lb # [ 167.422763] niks3-server[1152]: 2026/09/23 05:15:23 INFO Signed narinfos id=55 count=16324lb # [ 167.428956] niks3-server[1152]: 2026/09/23 05:15:23 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6325worker2 # [ 167.483580] nix-grpc-daemon[1797]: time=2026-09-23T05:15:24.047Z level=INFO msg="Upload complete. (240ms)"6326lb # [ 167.443203] niks3-server[1152]: 2026/09/23 05:15:24 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6327lb # [ 167.450277] niks3-server[1152]: 2026/09/23 05:15:24 INFO Completed upload id=556328lb # [ 167.453445] niks3-server[1152]: 2026/09/23 05:15:24 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6329worker2 # [ 167.496595] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=5286330worker2 # [ 167.511315] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826331worker2 # [ 167.517632] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6332worker2 # [ 167.522615] nix-daemon[858]: reaped child process 1874, status = succeeded6333client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6334worker2 # [ 167.534946] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6335client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6336lb # [ 168.370152] envoy[900]: conn peer=[2001:db8:1::1]:41662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=853 ms=35036337client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6338client # farm-a> LOG-farm-a-during-drain6339worker2 # [ 168.782535] nix-daemon[858]: reaped child process 1883, status = succeeded6340worker2 # [ 168.910712] nix-daemon[858]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6341worker2 # [ 168.929865] nix-daemon[858]: reaped child process 1902, status = succeeded6342lb # [ 168.893465] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received uploads request method=POST path=/api/pending_closures6343worker2 # [ 168.950772] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.512Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6344worker2 # [ 168.953402] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.512Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6345lb # [ 168.943934] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6346worker2 # [ 168.995379] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.559Z level=INFO msg="Uploading 1 narinfos"6347lb # [ 168.956410] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6348lb # [ 168.965592] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6349lb # [ 168.975888] niks3-server[1152]: 2026/09/23 05:15:25 INFO Signed narinfos id=56 count=16350worker2 # [ 169.018094] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.582Z level=INFO msg="Upload complete. (272ms)"6351lb # [ 168.979843] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6352lb # [ 168.984797] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6353lb # [ 168.990195] niks3-server[1152]: 2026/09/23 05:15:25 INFO Completed upload id=566354lb # [ 168.994353] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6355worker2 # [ 169.034358] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16356worker2 # [ 169.051939] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06357worker2 # [ 169.055646] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16358worker2 # [ 169.059498] nix-daemon[858]: reaped child process 1881, status = succeeded6359worker2 # [ 169.090782] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826360worker2 # [ 169.105975] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408826361client # copying 1 paths to 'lb:50051'...6362worker2 # [ 169.120653] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.684Z level=INFO msg="All 1 paths already cached"6363worker2 # [ 169.127714] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408826364client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6365worker2 # [ 169.149887] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6366worker2 # [ 169.303655] nix-daemon[858]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6367worker2 # [ 169.320803] nix-daemon[858]: reaped child process 1911, status = succeeded6368lb # [ 169.284114] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received uploads request method=POST path=/api/pending_closures6369worker2 # [ 169.335713] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.899Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6370worker2 # [ 169.337500] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.899Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6371worker2 # [ 169.361950] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.925Z level=INFO msg="Uploading 1 narinfos"6372lb # [ 169.312900] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6373lb # [ 169.325155] niks3-server[1152]: 2026/09/23 05:15:25 INFO Signed narinfos id=57 count=16374lb # [ 169.335213] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6375lb # [ 169.349770] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6376worker2 # [ 169.405242] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.969Z level=INFO msg="Upload complete. (238ms)"6377lb # [ 169.363388] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6378lb # [ 169.368826] niks3-server[1152]: 2026/09/23 05:15:25 INFO Completed upload id=576379lb # [ 169.372005] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6380worker2 # [ 169.414839] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=6406381worker2 # [ 169.429703] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826382worker2 # [ 169.436614] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6383worker2 # [ 169.441554] nix-daemon[858]: reaped child process 1906, status = succeeded6384client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6385worker2 # [ 169.454651] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6386client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6387client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6388client # farm-b> LOG-farm-b-during-drain6389worker2 # [ 170.712439] nix-daemon[858]: reaped child process 1915, status = succeeded6390worker2 # [ 170.800968] nix-daemon[858]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6391worker2 # [ 170.818454] nix-daemon[858]: reaped child process 1934, status = succeeded6392lb # [ 170.781384] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received uploads request method=POST path=/api/pending_closures6393worker2 # [ 170.834740] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.397Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6394worker2 # [ 170.836407] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.397Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6395lb # [ 170.816852] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6396lb # [ 170.831305] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6397worker2 # [ 170.882273] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.446Z level=INFO msg="Uploading 1 narinfos"6398lb # [ 170.842828] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6399lb # [ 170.853718] niks3-server[1152]: 2026/09/23 05:15:27 INFO Signed narinfos id=58 count=16400lb # [ 170.862636] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6401worker2 # [ 170.911998] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.476Z level=INFO msg="Upload complete. (247ms)"6402lb # [ 170.873275] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6403lb # [ 170.879142] niks3-server[1152]: 2026/09/23 05:15:27 INFO Completed upload id=586404lb # [ 170.882752] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6405worker2 # [ 170.927731] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16406worker2 # [ 170.948576] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06407worker2 # [ 170.954391] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16408worker2 # [ 170.959379] nix-daemon[858]: reaped child process 1913, status = succeeded6409worker2 # [ 170.990800] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826410worker2 # [ 171.005848] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408826411worker2 # [ 171.019757] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.583Z level=INFO msg="All 2 paths already cached"6412client # copying 1 paths to 'lb:50051'...6413client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6414worker2 # [ 171.031269] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408826415worker2 # [ 171.048975] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6416worker2 # [ 171.206213] nix-daemon[858]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6417worker2 # [ 171.224526] nix-daemon[858]: reaped child process 1943, status = succeeded6418lb # [ 171.185975] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received uploads request method=POST path=/api/pending_closures6419worker2 # [ 171.237022] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.800Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6420worker2 # [ 171.238660] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.800Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6421lb # [ 171.215592] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6422worker2 # [ 171.266309] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.830Z level=INFO msg="Uploading 1 narinfos"6423lb # [ 171.231161] niks3-server[1152]: 2026/09/23 05:15:27 INFO Signed narinfos id=59 count=16424lb # [ 171.238371] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6425lb # [ 171.250082] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6426worker2 # [ 171.300606] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.864Z level=INFO msg="Upload complete. (234ms)"6427lb # [ 171.261043] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6428lb # [ 171.266648] niks3-server[1152]: 2026/09/23 05:15:27 INFO Completed upload id=596429worker2 # [ 171.308318] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=7686430lb # [ 171.271348] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6431worker2 # [ 171.322286] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826432worker2 # [ 171.331192] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6433worker2 # [ 171.335040] nix-daemon[858]: reaped child process 1938, status = succeeded6434client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6435worker2 # [ 171.345631] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6436client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6437client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6438client # farm-top> LOG-farm-top-during-drain6439worker2 # [ 172.581544] nix-daemon[858]: reaped child process 1947, status = succeeded6440worker2 # [ 172.702177] nix-daemon[858]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6441worker2 # [ 172.717911] nix-daemon[858]: reaped child process 1966, status = succeeded6442lb # [ 172.683435] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received uploads request method=POST path=/api/pending_closures6443worker2 # [ 172.736409] nix-grpc-daemon[1797]: time=2026-09-23T05:15:29.299Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6444worker2 # [ 172.737945] nix-grpc-daemon[1797]: time=2026-09-23T05:15:29.299Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6445worker2 # [ 172.774747] nix-grpc-daemon[1797]: time=2026-09-23T05:15:29.338Z level=INFO msg="Uploading 1 narinfos"6446lb # [ 172.723208] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6447lb # [ 172.738779] niks3-server[1152]: 2026/09/23 05:15:29 INFO Signed narinfos id=60 count=16448lb # [ 172.749247] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6449lb # [ 172.764155] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6450lb # [ 172.767517] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6451lb # [ 172.773396] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6452worker2 # [ 172.814896] nix-grpc-daemon[1797]: time=2026-09-23T05:15:29.378Z level=INFO msg="Upload complete. (253ms)"6453lb # [ 172.776593] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6454lb # [ 172.780756] niks3-server[1152]: 2026/09/23 05:15:29 INFO Completed upload id=606455worker2 # [ 172.827422] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16456worker2 # [ 172.846099] nix-daemon[858]: reaped child process 1945, status = succeeded6457client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.00 seconds)6458client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6459client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6460worker1: must succeed: pkill -f 'read -t [9]0 x'6461worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6462client: waiting for success: ! systemctl is-active sw6463worker1 # [ 173.327984] nix-daemon[835]: reaped child process 1571, status = succeeded6464worker1 # [ 173.396780] nix-daemon[835]: accepted connection from pid 1613, user nix-grpc-daemon (trusted)6465worker1 # [ 173.414766] nix-daemon[835]: reaped child process 1616, status = succeeded6466lb # [ 173.376376] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received uploads request method=POST path=/api/pending_closures6467worker1 # [ 173.428367] nix-grpc-daemon[1516]: time=2026-09-23T05:15:29.983Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6468worker1 # [ 173.429985] nix-grpc-daemon[1516]: time=2026-09-23T05:15:29.984Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6469lb # [ 173.419300] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6470worker1 # [ 173.470801] nix-grpc-daemon[1516]: time=2026-09-23T05:15:30.026Z level=INFO msg="Uploading 1 narinfos"6471lb # [ 173.433734] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6472lb # [ 173.444541] niks3-server[1152]: 2026/09/23 05:15:29 INFO Signed narinfos id=61 count=16473lb # [ 173.452239] niks3-server[1152]: 2026/09/23 05:15:30 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6474worker1 # [ 173.501221] nix-grpc-daemon[1516]: time=2026-09-23T05:15:30.057Z level=INFO msg="Upload complete. (264ms)"6475lb # [ 173.461688] niks3-server[1152]: 2026/09/23 05:15:30 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6476lb # [ 173.467041] niks3-server[1152]: 2026/09/23 05:15:30 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6477lb # [ 173.472801] niks3-server[1152]: 2026/09/23 05:15:30 INFO Completed upload id=616478lb # [ 173.475772] niks3-server[1152]: 2026/09/23 05:15:30 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6479worker1 # [ 173.517149] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38116 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16480worker1 # [ 173.532784] nix-daemon[835]: reaped child process 1569, status = succeeded6481client # [ 173.550287] systemd[1]: sw.service: Deactivated successfully.6482client # [ 173.555543] systemd[1]: sw.service: Consumed 197ms CPU time over 9.392s wall clock time, 43.5M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic.6483worker1 # [ 174.114515] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:30Z level=info event=signal_exit6484client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6485client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6486client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6487worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6488worker1 # [ 176.120951] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:32Z level=info event=scheduler_restarting6489worker2 # [ 176.120325] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:408666490worker2 # [ 176.120637] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:32Z level=info event=worker_gone addr=192.168.1.3:500516491worker1 # [ 176.146553] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6492worker1 # [ 176.170304] nix-daemon[835]: reaped child process 1470, status = succeeded6493worker1 # [ 176.955039] nix-grpc-daemon[1413]: E0000 00:00:1790140533.511022 1459 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6494worker1 # [ 176.962852] nix-grpc-daemon[1413]: E0000 00:00:1790140533.518916 1459 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6495worker1 # [ 178.142782] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6496worker1 # [ 178.148548] systemd[1]: nix-grpc-daemon.service: Consumed 872ms CPU time over 30.232s wall clock time, 36M memory peak, 21.6K incoming IP traffic, 28.5K outgoing IP traffic.6497worker1 # [ 178.173052] systemd[1]: Starting Nix worker-protocol over gRPC...6498worker1 # [ 178.316720] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6499worker1 # [ 178.318737] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6500worker1 # [ 178.322790] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16501worker1 # [ 178.328049] nix-grpc-daemon[1630]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6502worker1 # [ 178.348179] nix-grpc-daemon[1630]: E0000 00:00:1790140534.883894 1630 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6503worker1 # [ 178.350522] nix-grpc-daemon[1630]: E0000 00:00:1790140534.887460 1630 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6504worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 4.11 seconds)6505lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6506lb: must succeed: curl -sf localhost:9901/clusters6507worker1 # [ 178.377671] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6508worker1 # [ 178.379834] nix-grpc-daemon[1630]: E0000 00:00:1790140534.894801 1673 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6509worker1 # [ 178.382561] systemd[1]: Started Nix worker-protocol over gRPC.6510worker1 # [ 178.383374] nix-grpc-daemon[1630]: E0000 00:00:1790140534.916489 1673 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6511worker1 # [ 178.385780] nix-grpc-daemon[1630]: E0000 00:00:1790140534.917748 1673 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6512worker1 # [ 178.388097] nix-grpc-daemon[1630]: E0000 00:00:1790140534.933184 1673 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6513worker1 # [ 178.396102] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=scheduler_connected addr=lb:500516514worker2 # [ 178.398700] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:408666515worker2 # [ 178.404602] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06516lb # [ 178.377285] envoy[900]: conn peer=[2001:db8:1::1]:53242 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4375 ms=57586517lb # [ 178.377612] envoy[900]: conn peer=[2001:db8:1::1]:53230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1537 ms=91636518lb # [ 178.377734] envoy[900]: conn peer=[2001:db8:1::3]:57540 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1020 tx=482 ms=279666519lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6520lb: must succeed: curl -sf localhost:9901/clusters6521lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6522lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)6523lb: must succeed: curl -sf localhost:9901/clusters6524lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6525worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6526worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6527(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.54 seconds)6528subtest: queue gauges show a burst while every build blocks6529lb: must succeed: curl -sf localhost:9901/clusters6530lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6531lb: must succeed: curl -sf localhost:9901/clusters6532lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6533worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6534worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6535client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6536client # [ 179.856803] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6537client # Running as unit: burst.service; invocation ID: c3bec4e0958d42ccb08dbf9526dc5b116538client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6539lb: must succeed: curl -sf localhost:9901/clusters6540lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6541lb: must succeed: curl -sf localhost:9901/clusters6542lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6543worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6544client # [ 180.049720] nix[1548]: don't know how to build these paths:6545client # [ 180.050694] nix[1548]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6546client # [ 180.051728] nix[1548]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6547worker2 # [ 180.049026] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:408826548client # [ 180.053377] nix[1548]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6549client # [ 180.054376] nix[1548]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6550worker2 # [ 180.064078] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06551worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6552worker2 # [ 180.071301] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.4:50051 clients=16553worker2 # [ 180.076502] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06554worker2 # [ 180.080860] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.3:50051 clients=16555worker2 # [ 180.085751] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06556worker2 # [ 180.089821] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06557worker2 # [ 180.115402] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826558worker1 # [ 180.122262] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066559worker2 # [ 180.131718] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408826560worker1 # [ 180.135987] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6561client # [ 180.147317] nix[1548]: copying 1 paths to 'lb:50051'...6562worker2 # [ 180.146591] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408826563client # [ 180.149872] nix[1548]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6564worker2 # [ 180.161408] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6565worker1 # [ 180.161786] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 1/5)6566worker1 # [ 180.234706] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)6567worker1 # [ 180.259268] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 363 ms (attempt 3/5)6568worker2 # [ 180.307194] nix-daemon[858]: accepted connection from pid 1981, user nix-grpc-daemon (trusted)6569worker2 # [ 180.327125] nix-daemon[858]: reaped child process 1984, status = succeeded6570lb # [ 180.291467] niks3-server[1152]: 2026/09/23 05:15:36 INFO Received uploads request method=POST path=/api/pending_closures6571worker2 # [ 180.344332] nix-grpc-daemon[1797]: time=2026-09-23T05:15:36.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6572worker2 # [ 180.345977] nix-grpc-daemon[1797]: time=2026-09-23T05:15:36.907Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6573lb # [ 180.329051] niks3-server[1152]: 2026/09/23 05:15:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6574worker2 # [ 180.379817] nix-grpc-daemon[1797]: time=2026-09-23T05:15:36.943Z level=INFO msg="Uploading 1 narinfos"6575lb # [ 180.340366] niks3-server[1152]: 2026/09/23 05:15:36 INFO Signed narinfos id=62 count=16576lb # [ 180.352214] niks3-server[1152]: 2026/09/23 05:15:36 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6577lb # [ 180.362379] niks3-server[1152]: 2026/09/23 05:15:36 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6578lb # [ 180.374163] niks3-server[1152]: 2026/09/23 05:15:36 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6579worker2 # [ 180.423169] nix-grpc-daemon[1797]: time=2026-09-23T05:15:36.987Z level=INFO msg="Upload complete. (246ms)"6580lb # [ 180.382720] niks3-server[1152]: 2026/09/23 05:15:36 INFO Completed upload id=626581lb # [ 180.386347] niks3-server[1152]: 2026/09/23 05:15:36 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6582worker2 # [ 180.437052] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=4166583worker2 # [ 180.452936] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826584worker2 # [ 180.463293] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6585worker2 # [ 180.466672] nix-daemon[858]: reaped child process 1979, status = succeeded6586client # [ 180.479178] nix[1548]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6587worker2 # [ 180.482897] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6588worker1 # [ 180.631976] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 185 ms (attempt 4/5)6589worker1 # [ 180.819816] nix-grpc-daemon[1630]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6590worker1 # [ 180.825787] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6591worker1 # [ 180.844613] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066592client # [ 180.860169] nix[1548]: copying 1 paths to 'lb:50051'...6593worker1 # [ 180.860888] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414066594client # [ 180.862508] nix[1548]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6595worker1 # [ 180.876442] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6596worker1 # [ 180.911191] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=info event=niks3_push_started pid=16916597worker1 # [ 181.051879] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.607Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6598lb: must succeed: curl -sf localhost:9901/clusters6599lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6600lb: must succeed: curl -sf localhost:9901/clusters6601worker1 # [ 181.162954] nix-daemon[835]: accepted connection from pid 1704, user nix-grpc-daemon (trusted)6602worker1 # [ 181.183522] nix-daemon[835]: reaped child process 1707, status = succeeded6603lb # [ 181.145544] niks3-server[1152]: 2026/09/23 05:15:37 INFO Received uploads request method=POST path=/api/pending_closures6604lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6605worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6606worker1 # [ 181.196250] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.752Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6607worker1 # [ 181.197681] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.752Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6608lb # [ 181.179639] niks3-server[1152]: 2026/09/23 05:15:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6609worker1 # [ 181.226746] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.782Z level=INFO msg="Uploading 1 narinfos"6610lb # [ 181.184126] niks3-server[1152]: 2026/09/23 05:15:37 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6611lb # [ 181.188705] niks3-server[1152]: 2026/09/23 05:15:37 INFO Signed narinfos id=63 count=16612lb # [ 181.192521] niks3-server[1152]: 2026/09/23 05:15:37 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6613lb # [ 181.202150] niks3-server[1152]: 2026/09/23 05:15:37 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6614worker1 # [ 181.247791] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.804Z level=INFO msg="Upload complete. (192ms)"6615lb # [ 181.206611] niks3-server[1152]: 2026/09/23 05:15:37 INFO Completed upload id=636616lb # [ 181.210378] niks3-server[1152]: 2026/09/23 05:15:37 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6617worker1 # [ 181.256716] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=4166618worker1 # [ 181.260291] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066619worker1 # [ 181.269494] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6620worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6621worker1 # [ 181.273478] nix-daemon[835]: reaped child process 1689, status = succeeded6622worker1 # [ 181.288224] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6623client # [ 181.286663] nix[1548]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6624worker2 # [ 182.718580] nix-daemon[858]: reaped child process 1988, status = succeeded6625worker2 # [ 182.820558] nix-daemon[858]: accepted connection from pid 2016, user nix-grpc-daemon (trusted)6626worker2 # [ 182.843117] nix-daemon[858]: reaped child process 2019, status = succeeded6627lb # [ 182.800610] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received uploads request method=POST path=/api/pending_closures6628worker2 # [ 182.852526] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.415Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6629worker2 # [ 182.854002] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.415Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6630lb # [ 182.845738] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6631worker2 # [ 182.896869] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.460Z level=INFO msg="Uploading 1 narinfos"6632lb # [ 182.859599] niks3-server[1152]: 2026/09/23 05:15:39 INFO Signed narinfos id=64 count=16633lb # [ 182.867448] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6634lb # [ 182.878035] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6635worker2 # [ 182.923607] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.487Z level=INFO msg="Upload complete. (227ms)"6636lb # [ 182.884564] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6637lb # [ 182.890397] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6638lb # [ 182.894098] niks3-server[1152]: 2026/09/23 05:15:39 INFO Completed upload id=646639lb # [ 182.895927] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6640worker2 # [ 182.937034] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=16641worker2 # [ 182.950515] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16642worker2 # [ 182.954558] nix-daemon[858]: reaped child process 1986, status = succeeded6643worker2 # [ 182.979009] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826644worker2 # [ 182.994782] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408826645client # [ 183.009411] nix[1548]: copying 1 paths to 'lb:50051'...6646worker2 # [ 183.009774] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408826647client # [ 183.014249] nix[1548]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6648worker2 # [ 183.023815] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6649worker2 # [ 183.170654] nix-daemon[858]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)6650worker2 # [ 183.191158] nix-daemon[858]: reaped child process 2026, status = succeeded6651lb # [ 183.147413] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received uploads request method=POST path=/api/pending_closures6652worker2 # [ 183.198824] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.762Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6653worker2 # [ 183.200550] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.762Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6654worker2 # [ 183.221639] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.785Z level=INFO msg="Uploading 1 narinfos"6655lb # [ 183.179538] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6656lb # [ 183.191371] niks3-server[1152]: 2026/09/23 05:15:39 INFO Signed narinfos id=65 count=16657lb # [ 183.201304] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6658lb # [ 183.216785] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6659worker2 # [ 183.266942] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.830Z level=INFO msg="Upload complete. (228ms)"6660lb # [ 183.226140] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6661lb # [ 183.231766] niks3-server[1152]: 2026/09/23 05:15:39 INFO Completed upload id=656662worker2 # [ 183.276304] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=4166663lb # [ 183.236733] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6664worker2 # [ 183.294146] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408826665worker2 # [ 183.301243] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6666worker2 # [ 183.306399] nix-daemon[858]: reaped child process 2021, status = succeeded6667worker2 # [ 183.317725] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6668client # [ 183.317195] nix[1548]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6669worker1 # [ 183.999877] nix-daemon[835]: reaped child process 1714, status = succeeded6670worker1 # [ 184.077504] nix-daemon[835]: accepted connection from pid 1740, user nix-grpc-daemon (trusted)6671worker1 # [ 184.096461] nix-daemon[835]: reaped child process 1743, status = succeeded6672lb # [ 184.058340] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received uploads request method=POST path=/api/pending_closures6673worker1 # [ 184.112003] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.665Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6674worker1 # [ 184.114377] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.666Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6675lb # [ 184.096697] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6676worker1 # [ 184.147839] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.703Z level=INFO msg="Uploading 1 narinfos"6677lb # [ 184.112722] niks3-server[1152]: 2026/09/23 05:15:40 INFO Signed narinfos id=66 count=16678lb # [ 184.121748] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6679lb # [ 184.134076] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6680worker1 # [ 184.188137] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.744Z level=INFO msg="Upload complete. (282ms)"6681lb # [ 184.145968] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6682lb # [ 184.150480] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6683lb # [ 184.155410] niks3-server[1152]: 2026/09/23 05:15:40 INFO Completed upload id=666684lb # [ 184.158349] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6685worker2 # [ 184.199553] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:40Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16686worker1 # [ 184.201126] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=16687worker1 # [ 184.218226] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066688worker1 # [ 184.223864] nix-daemon[835]: reaped child process 1709, status = succeeded6689worker1 # [ 184.231440] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066690client # [ 184.245326] nix[1548]: copying 1 paths to 'lb:50051'...6691worker1 # [ 184.243793] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414066692client # [ 184.249312] nix[1548]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6693worker1 # [ 184.256289] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6694worker1 # [ 184.393408] nix-daemon[835]: accepted connection from pid 1747, user nix-grpc-daemon (trusted)6695worker1 # [ 184.410446] nix-daemon[835]: reaped child process 1750, status = succeeded6696lb # [ 184.373613] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received uploads request method=POST path=/api/pending_closures6697worker1 # [ 184.424064] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.980Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6698worker1 # [ 184.425514] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.980Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6699lb # [ 184.404176] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6700worker1 # [ 184.454118] nix-grpc-daemon[1691]: time=2026-09-23T05:15:41.010Z level=INFO msg="Uploading 1 narinfos"6701lb # [ 184.419561] niks3-server[1152]: 2026/09/23 05:15:40 INFO Signed narinfos id=67 count=16702lb # [ 184.426324] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6703lb # [ 184.436918] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6704worker1 # [ 184.485138] nix-grpc-daemon[1691]: time=2026-09-23T05:15:41.041Z level=INFO msg="Upload complete. (216ms)"6705lb # [ 184.443706] niks3-server[1152]: 2026/09/23 05:15:41 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6706lb # [ 184.448681] niks3-server[1152]: 2026/09/23 05:15:41 INFO Completed upload id=676707worker1 # [ 184.495288] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=4166708lb # [ 184.454639] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6709worker1 # [ 184.507476] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066710worker1 # [ 184.517365] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6711worker1 # [ 184.520478] nix-daemon[835]: reaped child process 1745, status = succeeded6712client # [ 184.530919] nix[1548]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6713worker1 # [ 184.533649] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6714worker2 # [ 185.185528] nix-daemon[858]: reaped child process 2030, status = succeeded6715worker2 # [ 185.309429] nix-daemon[858]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)6716worker2 # [ 185.325502] nix-daemon[858]: reaped child process 2058, status = succeeded6717lb # [ 185.290693] niks3-server[1152]: 2026/09/23 05:15:41 INFO Received uploads request method=POST path=/api/pending_closures6718worker2 # [ 185.344456] nix-grpc-daemon[1797]: time=2026-09-23T05:15:41.905Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6719worker2 # [ 185.346302] nix-grpc-daemon[1797]: time=2026-09-23T05:15:41.906Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6720lb # [ 185.339755] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6721worker2 # [ 185.393173] nix-grpc-daemon[1797]: time=2026-09-23T05:15:41.956Z level=INFO msg="Uploading 1 narinfos"6722lb # [ 185.352224] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6723lb # [ 185.366299] niks3-server[1152]: 2026/09/23 05:15:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6724lb # [ 185.378565] niks3-server[1152]: 2026/09/23 05:15:41 INFO Signed narinfos id=68 count=16725lb # [ 185.384618] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6726worker2 # [ 185.424757] nix-grpc-daemon[1797]: time=2026-09-23T05:15:41.988Z level=INFO msg="Upload complete. (291ms)"6727lb # [ 185.390757] niks3-server[1152]: 2026/09/23 05:15:41 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6728lb # [ 185.394636] niks3-server[1152]: 2026/09/23 05:15:41 INFO Completed upload id=686729lb # [ 185.397239] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6730worker2 # [ 185.437317] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=16731worker2 # [ 185.455087] nix-daemon[858]: reaped child process 2028, status = succeeded6732worker1 # [ 186.157992] nix-daemon[835]: reaped child process 1754, status = succeeded6733worker1 # [ 186.262730] nix-daemon[835]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6734worker1 # [ 186.281807] nix-daemon[835]: reaped child process 1782, status = succeeded6735lb # [ 186.244155] niks3-server[1152]: 2026/09/23 05:15:42 INFO Received uploads request method=POST path=/api/pending_closures6736worker1 # [ 186.296408] nix-grpc-daemon[1691]: time=2026-09-23T05:15:42.851Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6737worker1 # [ 186.297887] nix-grpc-daemon[1691]: time=2026-09-23T05:15:42.851Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6738lb # [ 186.284376] niks3-server[1152]: 2026/09/23 05:15:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6739worker1 # [ 186.332652] nix-grpc-daemon[1691]: time=2026-09-23T05:15:42.888Z level=INFO msg="Uploading 1 narinfos"6740lb # [ 186.300816] niks3-server[1152]: 2026/09/23 05:15:42 INFO Signed narinfos id=69 count=16741lb # [ 186.303384] niks3-server[1152]: 2026/09/23 05:15:42 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6742worker1 # [ 186.361606] nix-grpc-daemon[1691]: time=2026-09-23T05:15:42.917Z level=INFO msg="Upload complete. (218ms)"6743lb # [ 186.320485] niks3-server[1152]: 2026/09/23 05:15:42 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6744lb # [ 186.323199] niks3-server[1152]: 2026/09/23 05:15:42 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6745lb # [ 186.328272] niks3-server[1152]: 2026/09/23 05:15:42 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6746lb # [ 186.331606] niks3-server[1152]: 2026/09/23 05:15:42 INFO Completed upload id=696747worker1 # [ 186.371263] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=16748lb # [ 186.334343] niks3-server[1152]: 2026/09/23 05:15:42 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6749worker1 # [ 186.385545] nix-daemon[835]: reaped child process 1752, status = succeeded6750client # [ 186.411514] systemd[1]: burst.service: Deactivated successfully.6751client # [ 186.415831] systemd[1]: burst.service: Consumed 219ms CPU time over 6.558s wall clock time, 51.6M memory peak, 14K incoming IP traffic, 18.9K outgoing IP traffic.6752client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6753client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6754lb: must succeed: curl -sf localhost:9901/clusters6755lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6756lb: must succeed: curl -sf localhost:9901/clusters6757lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6758worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6759worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6760(finished: subtest: queue gauges show a burst while every build blocks, in 8.06 seconds)6761subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6762client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16763worker1 # [ 187.909616] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414066764worker1 # [ 187.919781] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066765worker2 # [ 187.964696] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:44Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06766worker2 # [ 187.970131] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:44Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16767worker2 # [ 187.976207] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:44Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06768worker2 # [ 187.981621] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:44Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06769worker1 # [ 188.011009] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066770worker1 # [ 188.026602] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066771worker1 # [ 188.041934] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414066772worker1 # [ 188.057104] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6773worker1 # [ 188.216827] nix-daemon[835]: accepted connection from pid 1790, user nix-grpc-daemon (trusted)6774worker1 # [ 188.239353] nix-daemon[835]: reaped child process 1793, status = succeeded6775lb # [ 188.197504] niks3-server[1152]: 2026/09/23 05:15:44 INFO Received uploads request method=POST path=/api/pending_closures6776worker1 # [ 188.250488] nix-grpc-daemon[1691]: time=2026-09-23T05:15:44.806Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6777worker1 # [ 188.252082] nix-grpc-daemon[1691]: time=2026-09-23T05:15:44.806Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6778lb # [ 188.230714] niks3-server[1152]: 2026/09/23 05:15:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6779lb # [ 188.247897] niks3-server[1152]: 2026/09/23 05:15:44 INFO Signed narinfos id=70 count=16780worker1 # [ 188.290828] nix-grpc-daemon[1691]: time=2026-09-23T05:15:44.846Z level=INFO msg="Uploading 1 narinfos"6781lb # [ 188.259413] niks3-server[1152]: 2026/09/23 05:15:44 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6782lb # [ 188.276375] niks3-server[1152]: 2026/09/23 05:15:44 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6783lb # [ 188.290580] niks3-server[1152]: 2026/09/23 05:15:44 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6784lb # [ 188.297675] niks3-server[1152]: 2026/09/23 05:15:44 INFO Completed upload id=706785worker1 # [ 188.338310] nix-grpc-daemon[1691]: time=2026-09-23T05:15:44.894Z level=INFO msg="Upload complete. (264ms)"6786lb # [ 188.300413] niks3-server[1152]: 2026/09/23 05:15:44 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6787worker1 # [ 188.349833] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=5126788worker1 # [ 188.368079] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066789worker1 # [ 188.375531] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6790worker1 # [ 188.381932] nix-daemon[835]: reaped child process 1788, status = succeeded6791worker1 # [ 188.395256] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6792lb # [ 188.373323] envoy[900]: conn peer=[2001:db8:1::1]:58694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9688 tx=5208 ms=63456793worker1 # [ 189.439259] nix-daemon[835]: reaped child process 1797, status = succeeded6794worker1 # [ 189.529119] nix-daemon[835]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)6795worker1 # [ 189.549938] nix-daemon[835]: reaped child process 1816, status = succeeded6796lb # [ 189.506434] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received uploads request method=POST path=/api/pending_closures6797worker1 # [ 189.558988] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.113Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6798worker1 # [ 189.560683] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.113Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6799lb # [ 189.543869] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6800worker1 # [ 189.598485] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.154Z level=INFO msg="Uploading 1 narinfos"6801lb # [ 189.557461] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6802lb # [ 189.567060] niks3-server[1152]: 2026/09/23 05:15:46 INFO Signed narinfos id=71 count=16803lb # [ 189.572500] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6804worker1 # [ 189.628102] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.184Z level=INFO msg="Upload complete. (217ms)"6805lb # [ 189.586236] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6806lb # [ 189.594292] niks3-server[1152]: 2026/09/23 05:15:46 INFO Completed upload id=716807worker1 # [ 189.637207] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=16808lb # [ 189.597764] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6809worker2 # [ 189.638809] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:46Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16810lb # [ 189.602353] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6811worker1 # [ 189.652523] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066812worker2 # [ 189.650860] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:46Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06813worker1 # [ 189.658503] nix-daemon[835]: reaped child process 1795, status = succeeded6814worker1 # [ 189.665452] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066815worker1 # [ 189.677025] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414066816worker1 # [ 189.686615] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6817worker1 # [ 189.815344] nix-daemon[835]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6818worker1 # [ 189.836473] nix-daemon[835]: reaped child process 1823, status = succeeded6819lb # [ 189.796564] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received uploads request method=POST path=/api/pending_closures6820worker1 # [ 189.847933] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.403Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6821worker1 # [ 189.849649] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.403Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6822lb # [ 189.828985] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6823worker1 # [ 189.878563] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.434Z level=INFO msg="Uploading 1 narinfos"6824lb # [ 189.843846] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6825worker1 # [ 189.899097] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.455Z level=INFO msg="Upload complete. (202ms)"6826lb # [ 189.857859] niks3-server[1152]: 2026/09/23 05:15:46 INFO Signed narinfos id=72 count=16827lb # [ 189.864225] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6828worker1 # [ 189.906909] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=5126829lb # [ 189.867550] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6830lb # [ 189.872460] niks3-server[1152]: 2026/09/23 05:15:46 INFO Completed upload id=726831lb # [ 189.875124] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6832worker1 # [ 189.919037] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066833worker1 # [ 189.924654] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6834worker1 # [ 189.928959] nix-daemon[835]: reaped child process 1818, status = succeeded6835worker1 # [ 189.937912] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6836worker1 # [ 191.173342] nix-daemon[835]: reaped child process 1827, status = succeeded6837worker1 # [ 191.285184] nix-daemon[835]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6838worker1 # [ 191.304839] nix-daemon[835]: reaped child process 1846, status = succeeded6839lb # [ 191.267599] niks3-server[1152]: 2026/09/23 05:15:47 INFO Received uploads request method=POST path=/api/pending_closures6840worker1 # [ 191.320374] nix-grpc-daemon[1691]: time=2026-09-23T05:15:47.875Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6841worker1 # [ 191.321821] nix-grpc-daemon[1691]: time=2026-09-23T05:15:47.875Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6842lb # [ 191.308916] niks3-server[1152]: 2026/09/23 05:15:47 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6843worker1 # [ 191.364929] nix-grpc-daemon[1691]: time=2026-09-23T05:15:47.921Z level=INFO msg="Uploading 1 narinfos"6844lb # [ 191.324004] niks3-server[1152]: 2026/09/23 05:15:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6845lb # [ 191.339009] niks3-server[1152]: 2026/09/23 05:15:47 INFO Signed narinfos id=73 count=16846lb # [ 191.346396] niks3-server[1152]: 2026/09/23 05:15:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6847worker1 # [ 191.400727] nix-grpc-daemon[1691]: time=2026-09-23T05:15:47.956Z level=INFO msg="Upload complete. (252ms)"6848lb # [ 191.354283] niks3-server[1152]: 2026/09/23 05:15:47 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6849lb # [ 191.364566] niks3-server[1152]: 2026/09/23 05:15:47 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6850lb # [ 191.369472] niks3-server[1152]: 2026/09/23 05:15:47 INFO Completed upload id=736851worker2 # [ 191.408805] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:47Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16852lb # [ 191.372910] niks3-server[1152]: 2026/09/23 05:15:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6853worker1 # [ 191.413248] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=16854worker2 # [ 191.422669] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:47Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06855worker1 # [ 191.429796] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066856worker1 # [ 191.435897] nix-daemon[835]: reaped child process 1825, status = succeeded6857worker1 # [ 191.443530] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066858worker1 # [ 191.455202] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414066859worker1 # [ 191.467237] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6860worker1 # [ 191.600211] nix-daemon[835]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6861worker1 # [ 191.618982] nix-daemon[835]: reaped child process 1853, status = succeeded6862lb # [ 191.582900] niks3-server[1152]: 2026/09/23 05:15:48 INFO Received uploads request method=POST path=/api/pending_closures6863worker1 # [ 191.633760] nix-grpc-daemon[1691]: time=2026-09-23T05:15:48.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6864worker1 # [ 191.635440] nix-grpc-daemon[1691]: time=2026-09-23T05:15:48.189Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6865worker1 # [ 191.657079] nix-grpc-daemon[1691]: time=2026-09-23T05:15:48.213Z level=INFO msg="Uploading 1 narinfos"6866lb # [ 191.615346] niks3-server[1152]: 2026/09/23 05:15:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6867lb # [ 191.628736] niks3-server[1152]: 2026/09/23 05:15:48 INFO Signed narinfos id=74 count=16868lb # [ 191.631833] niks3-server[1152]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6869lb # [ 191.641555] niks3-server[1152]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6870lb # [ 191.652743] niks3-server[1152]: 2026/09/23 05:15:48 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6871lb # [ 191.657369] niks3-server[1152]: 2026/09/23 05:15:48 INFO Completed upload id=746872worker1 # [ 191.693546] nix-grpc-daemon[1691]: time=2026-09-23T05:15:48.249Z level=INFO msg="Upload complete. (214ms)"6873lb # [ 191.661209] niks3-server[1152]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6874worker1 # [ 191.703926] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=5126875worker1 # [ 191.712440] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066876worker1 # [ 191.722301] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6877worker1 # [ 191.727105] nix-daemon[835]: reaped child process 1848, status = succeeded6878worker1 # [ 191.738990] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6879worker1 # [ 193.210250] nix-daemon[835]: reaped child process 1857, status = succeeded6880worker1 # [ 193.341322] nix-daemon[835]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6881worker1 # [ 193.361432] nix-daemon[835]: reaped child process 1876, status = succeeded6882lb # [ 193.317964] niks3-server[1152]: 2026/09/23 05:15:49 INFO Received uploads request method=POST path=/api/pending_closures6883worker1 # [ 193.371566] nix-grpc-daemon[1691]: time=2026-09-23T05:15:49.926Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6884worker1 # [ 193.373444] nix-grpc-daemon[1691]: time=2026-09-23T05:15:49.926Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6885lb # [ 193.359416] niks3-server[1152]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6886worker1 # [ 193.413658] nix-grpc-daemon[1691]: time=2026-09-23T05:15:49.969Z level=INFO msg="Uploading 1 narinfos"6887lb # [ 193.372930] niks3-server[1152]: 2026/09/23 05:15:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6888lb # [ 193.382538] niks3-server[1152]: 2026/09/23 05:15:49 INFO Signed narinfos id=75 count=16889lb # [ 193.390061] niks3-server[1152]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6890lb # [ 193.397457] niks3-server[1152]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6891worker1 # [ 193.439028] nix-grpc-daemon[1691]: time=2026-09-23T05:15:49.995Z level=INFO msg="Upload complete. (285ms)"6892lb # [ 193.400182] niks3-server[1152]: 2026/09/23 05:15:49 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6893lb # [ 193.404363] niks3-server[1152]: 2026/09/23 05:15:49 INFO Completed upload id=756894lb # [ 193.407128] niks3-server[1152]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6895worker2 # [ 193.448580] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:50Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16896worker1 # [ 193.451107] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=16897worker2 # [ 193.461178] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:50Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06898worker1 # [ 193.472479] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066899worker1 # [ 193.479031] nix-daemon[835]: reaped child process 1855, status = succeeded6900worker1 # [ 193.486701] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066901worker1 # [ 193.498838] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.055Z level=INFO msg="All 1 paths already cached"6902worker1 # [ 193.507989] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414066903worker1 # [ 193.522944] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6904worker1 # [ 193.661565] nix-daemon[835]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6905worker1 # [ 193.681578] nix-daemon[835]: reaped child process 1883, status = succeeded6906lb # [ 193.644333] niks3-server[1152]: 2026/09/23 05:15:50 INFO Received uploads request method=POST path=/api/pending_closures6907worker1 # [ 193.695322] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.251Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6908worker1 # [ 193.696965] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.251Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6909lb # [ 193.674215] niks3-server[1152]: 2026/09/23 05:15:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6910worker1 # [ 193.722421] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.278Z level=INFO msg="Uploading 1 narinfos"6911lb # [ 193.688894] niks3-server[1152]: 2026/09/23 05:15:50 INFO Signed narinfos id=76 count=16912lb # [ 193.696470] niks3-server[1152]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6913lb # [ 193.702704] niks3-server[1152]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6914worker1 # [ 193.754500] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.310Z level=INFO msg="Upload complete. (221ms)"6915lb # [ 193.712892] niks3-server[1152]: 2026/09/23 05:15:50 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6916lb # [ 193.718239] niks3-server[1152]: 2026/09/23 05:15:50 INFO Completed upload id=766917lb # [ 193.721741] niks3-server[1152]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6918worker1 # [ 193.764802] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=6246919worker1 # [ 193.775442] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066920worker1 # [ 193.782133] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6921worker1 # [ 193.786608] nix-daemon[835]: reaped child process 1878, status = succeeded6922worker1 # [ 193.797954] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6923worker1 # [ 194.634515] nix-daemon[835]: reaped child process 1887, status = succeeded6924worker1 # [ 194.744446] nix-daemon[835]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)6925worker1 # [ 194.762484] nix-daemon[835]: reaped child process 1906, status = succeeded6926lb # [ 194.727861] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received uploads request method=POST path=/api/pending_closures6927worker1 # [ 194.780498] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.336Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6928worker1 # [ 194.782008] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.336Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6929lb # [ 194.770180] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6930worker1 # [ 194.822337] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.378Z level=INFO msg="Uploading 1 narinfos"6931lb # [ 194.786383] niks3-server[1152]: 2026/09/23 05:15:51 INFO Signed narinfos id=77 count=16932lb # [ 194.793045] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6933lb # [ 194.801347] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6934lb # [ 194.809001] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6935lb # [ 194.812336] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6936lb # [ 194.817363] niks3-server[1152]: 2026/09/23 05:15:51 INFO Completed upload id=776937worker1 # [ 194.858628] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.414Z level=INFO msg="Upload complete. (249ms)"6938lb # [ 194.819907] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6939worker2 # [ 194.871565] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:51Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16940worker1 # [ 194.873726] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=16941worker2 # [ 194.881638] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:51Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06942worker1 # [ 194.888454] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066943worker1 # [ 194.899810] nix-daemon[835]: reaped child process 1885, status = succeeded6944worker1 # [ 194.904555] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066945worker1 # [ 194.913174] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.469Z level=INFO msg="All 1 paths already cached"6946worker1 # [ 194.921628] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414066947worker1 # [ 194.935697] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6948worker1 # [ 195.072436] nix-daemon[835]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6949worker1 # [ 195.091681] nix-daemon[835]: reaped child process 1913, status = succeeded6950lb # [ 195.054473] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received uploads request method=POST path=/api/pending_closures6951worker1 # [ 195.105118] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.661Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6952worker1 # [ 195.107097] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.663Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6953lb # [ 195.086150] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6954worker1 # [ 195.135503] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.691Z level=INFO msg="Uploading 1 narinfos"6955lb # [ 195.100888] niks3-server[1152]: 2026/09/23 05:15:51 INFO Signed narinfos id=78 count=16956lb # [ 195.108490] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6957lb # [ 195.121797] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6958worker1 # [ 195.169674] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.725Z level=INFO msg="Upload complete. (223ms)"6959lb # [ 195.129659] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6960lb # [ 195.135155] niks3-server[1152]: 2026/09/23 05:15:51 INFO Completed upload id=786961worker1 # [ 195.180303] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=6246962lb # [ 195.139784] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6963worker1 # [ 195.196743] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066964worker1 # [ 195.203282] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6965worker1 # [ 195.209013] nix-daemon[835]: reaped child process 1908, status = succeeded6966worker1 # [ 195.224469] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6967worker1 # [ 196.292186] nix-daemon[835]: reaped child process 1917, status = succeeded6968worker1 # [ 196.386672] nix-daemon[835]: accepted connection from pid 1933, user nix-grpc-daemon (trusted)6969worker1 # [ 196.405329] nix-daemon[835]: reaped child process 1936, status = succeeded6970lb # [ 196.369417] niks3-server[1152]: 2026/09/23 05:15:52 INFO Received uploads request method=POST path=/api/pending_closures6971worker1 # [ 196.424448] nix-grpc-daemon[1691]: time=2026-09-23T05:15:52.978Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6972worker1 # [ 196.426165] nix-grpc-daemon[1691]: time=2026-09-23T05:15:52.978Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6973lb # [ 196.419373] niks3-server[1152]: 2026/09/23 05:15:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6974worker1 # [ 196.472634] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.028Z level=INFO msg="Uploading 1 narinfos"6975lb # [ 196.432003] niks3-server[1152]: 2026/09/23 05:15:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6976lb # [ 196.444713] niks3-server[1152]: 2026/09/23 05:15:52 INFO Signed narinfos id=79 count=16977lb # [ 196.452466] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6978lb # [ 196.464945] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6979worker1 # [ 196.507459] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.063Z level=INFO msg="Upload complete. (272ms)"6980lb # [ 196.470713] niks3-server[1152]: 2026/09/23 05:15:53 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6981lb # [ 196.476112] niks3-server[1152]: 2026/09/23 05:15:53 INFO Completed upload id=796982worker2 # [ 196.519568] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:53Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16983lb # [ 196.480815] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6984worker1 # [ 196.522332] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=16985worker2 # [ 196.532920] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:53Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06986worker1 # [ 196.541862] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414066987worker1 # [ 196.548890] nix-daemon[835]: reaped child process 1915, status = succeeded6988worker1 # [ 196.556957] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414066989worker1 # [ 196.569960] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.126Z level=INFO msg="All 1 paths already cached"6990worker1 # [ 196.579461] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414066991worker1 # [ 196.594909] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6992worker1 # [ 196.732710] nix-daemon[835]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6993worker1 # [ 196.753959] nix-daemon[835]: reaped child process 1943, status = succeeded6994lb # [ 196.715024] niks3-server[1152]: 2026/09/23 05:15:53 INFO Received uploads request method=POST path=/api/pending_closures6995worker1 # [ 196.766187] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.322Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6996worker1 # [ 196.767831] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.322Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6997worker1 # [ 196.793960] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.349Z level=INFO msg="Uploading 1 narinfos"6998lb # [ 196.744654] niks3-server[1152]: 2026/09/23 05:15:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6999lb # [ 196.760203] niks3-server[1152]: 2026/09/23 05:15:53 INFO Signed narinfos id=80 count=17000lb # [ 196.763376] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst7001lb # [ 196.775269] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls7002worker1 # [ 196.827984] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.384Z level=INFO msg="Upload complete. (222ms)"7003lb # [ 196.784813] niks3-server[1152]: 2026/09/23 05:15:53 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7004lb # [ 196.791312] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo7005worker1 # [ 196.835010] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=6247006lb # [ 196.798898] niks3-server[1152]: 2026/09/23 05:15:53 INFO Completed upload id=807007worker1 # [ 196.849769] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414067008worker1 # [ 196.856451] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7009worker1 # [ 196.861151] nix-daemon[835]: reaped child process 1938, status = succeeded7010worker1 # [ 196.874645] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7011worker1 # [ 198.381789] nix-daemon[835]: reaped child process 1947, status = succeeded7012worker1 # [ 198.456113] nix-daemon[835]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)7013worker1 # [ 198.474538] nix-daemon[835]: reaped child process 1966, status = succeeded7014lb # [ 198.437601] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received uploads request method=POST path=/api/pending_closures7015worker1 # [ 198.492456] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.047Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7016worker1 # [ 198.494224] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.047Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"7017lb # [ 198.491222] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst7018worker1 # [ 198.539622] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.095Z level=INFO msg="Uploading 1 narinfos"7019lb # [ 198.503256] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7020lb # [ 198.512661] niks3-server[1152]: 2026/09/23 05:15:55 INFO Signed narinfos id=81 count=17021lb # [ 198.514492] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv7022lb # [ 198.519602] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls7023lb # [ 198.528356] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7024worker1 # [ 198.569816] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.126Z level=INFO msg="Upload complete. (283ms)"7025lb # [ 198.531941] niks3-server[1152]: 2026/09/23 05:15:55 INFO Completed upload id=817026lb # [ 198.533973] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo7027worker1 # [ 198.581337] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=17028worker2 # [ 198.585927] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:55Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=17029worker2 # [ 198.594344] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:55Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=07030worker1 # [ 198.603947] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414067031worker1 # [ 198.611011] nix-daemon[835]: reaped child process 1945, status = succeeded7032worker1 # [ 198.619171] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414067033worker1 # [ 198.633122] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.189Z level=INFO msg="All 2 paths already cached"7034worker1 # [ 198.642675] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414067035worker1 # [ 198.657392] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7036worker1 # [ 198.801255] nix-daemon[835]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)7037worker1 # [ 198.822156] nix-daemon[835]: reaped child process 1973, status = succeeded7038lb # [ 198.778413] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received uploads request method=POST path=/api/pending_closures7039worker1 # [ 198.830226] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.386Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7040worker1 # [ 198.831932] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.386Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"7041lb # [ 198.810799] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst7042worker1 # [ 198.863825] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.419Z level=INFO msg="Uploading 1 narinfos"7043lb # [ 198.824696] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7044lb # [ 198.833804] niks3-server[1152]: 2026/09/23 05:15:55 INFO Signed narinfos id=82 count=17045lb # [ 198.841156] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls7046worker1 # [ 198.888207] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.444Z level=INFO msg="Upload complete. (218ms)"7047lb # [ 198.848137] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7048lb # [ 198.852269] niks3-server[1152]: 2026/09/23 05:15:55 INFO Completed upload id=827049lb # [ 198.855096] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo7050worker1 # [ 198.896122] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=7527051worker1 # [ 198.911388] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414067052worker1 # [ 198.917400] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7053worker1 # [ 198.922211] nix-daemon[835]: reaped child process 1968, status = succeeded7054worker1 # [ 198.934598] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7055worker1 # [ 200.008254] nix-daemon[835]: reaped child process 1977, status = succeeded7056worker1 # [ 200.114078] nix-daemon[835]: accepted connection from pid 1993, user nix-grpc-daemon (trusted)7057worker1 # [ 200.129803] nix-daemon[835]: reaped child process 1996, status = succeeded7058lb # [ 200.095655] niks3-server[1152]: 2026/09/23 05:15:56 INFO Received uploads request method=POST path=/api/pending_closures7059worker1 # [ 200.151655] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.704Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7060worker1 # [ 200.153542] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.704Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"7061lb # [ 200.135826] niks3-server[1152]: 2026/09/23 05:15:56 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv7062lb # [ 200.148595] niks3-server[1152]: 2026/09/23 05:15:56 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst7063worker1 # [ 200.201133] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.757Z level=INFO msg="Uploading 1 narinfos"7064lb # [ 200.162644] niks3-server[1152]: 2026/09/23 05:15:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7065lb # [ 200.170234] niks3-server[1152]: 2026/09/23 05:15:56 INFO Signed narinfos id=83 count=17066lb # [ 200.177254] niks3-server[1152]: 2026/09/23 05:15:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls7067worker1 # [ 200.218509] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.774Z level=INFO msg="Upload complete. (273ms)"7068lb # [ 200.181845] niks3-server[1152]: 2026/09/23 05:15:56 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7069lb # [ 200.186234] niks3-server[1152]: 2026/09/23 05:15:56 INFO Completed upload id=837070lb # [ 200.188783] niks3-server[1152]: 2026/09/23 05:15:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo7071worker2 # [ 200.230194] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:56Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=17072worker1 # [ 200.232126] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=17073worker1 # [ 200.247539] nix-daemon[835]: reaped child process 1975, status = succeeded7074worker1 # [ 200.285094] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414067075worker1 # [ 200.300139] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414067076worker1 # [ 200.314025] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.870Z level=INFO msg="All 2 paths already cached"7077worker1 # [ 200.324119] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414067078worker1 # [ 200.336489] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7079worker1 # [ 200.486362] nix-daemon[835]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)7080worker1 # [ 200.504719] nix-daemon[835]: reaped child process 2003, status = succeeded7081lb # [ 200.467794] niks3-server[1152]: 2026/09/23 05:15:57 INFO Received uploads request method=POST path=/api/pending_closures7082worker1 # [ 200.519063] nix-grpc-daemon[1691]: time=2026-09-23T05:15:57.075Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7083worker1 # [ 200.520739] nix-grpc-daemon[1691]: time=2026-09-23T05:15:57.075Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"7084lb # [ 200.496288] niks3-server[1152]: 2026/09/23 05:15:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7085worker1 # [ 200.544913] nix-grpc-daemon[1691]: time=2026-09-23T05:15:57.101Z level=INFO msg="Uploading 1 narinfos"7086lb # [ 200.510598] niks3-server[1152]: 2026/09/23 05:15:57 INFO Signed narinfos id=84 count=17087lb # [ 200.517637] niks3-server[1152]: 2026/09/23 05:15:57 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls7088worker1 # [ 200.573271] nix-grpc-daemon[1691]: time=2026-09-23T05:15:57.129Z level=INFO msg="Upload complete. (217ms)"7089lb # [ 200.530121] niks3-server[1152]: 2026/09/23 05:15:57 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst7090worker1 # [ 200.580155] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=7527091lb # [ 200.543316] niks3-server[1152]: 2026/09/23 05:15:57 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7092lb # [ 200.548795] niks3-server[1152]: 2026/09/23 05:15:57 INFO Completed upload id=847093worker1 # [ 200.591211] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414067094lb # [ 200.551269] niks3-server[1152]: 2026/09/23 05:15:57 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo7095worker1 # [ 200.601342] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7096worker1 # [ 200.607884] nix-daemon[835]: reaped child process 1998, status = succeeded7097worker1 # [ 200.620343] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7098worker1 # [ 201.927725] nix-daemon[835]: reaped child process 2007, status = succeeded7099worker1 # [ 202.000610] nix-daemon[835]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)7100worker1 # [ 202.018240] nix-daemon[835]: reaped child process 2026, status = succeeded7101lb # [ 201.981424] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received uploads request method=POST path=/api/pending_closures7102worker1 # [ 202.034719] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.589Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7103worker1 # [ 202.036546] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.589Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"7104lb # [ 202.026339] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7105worker1 # [ 202.077983] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.633Z level=INFO msg="Uploading 1 narinfos"7106lb # [ 202.041254] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv7107lb # [ 202.049222] niks3-server[1152]: 2026/09/23 05:15:58 INFO Signed narinfos id=85 count=17108lb # [ 202.053904] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls7109worker1 # [ 202.105038] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.661Z level=INFO msg="Upload complete. (274ms)"7110lb # [ 202.062788] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst7111lb # [ 202.066788] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7112lb # [ 202.074117] niks3-server[1152]: 2026/09/23 05:15:58 INFO Completed upload id=857113worker2 # [ 202.115684] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:58Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=17114lb # [ 202.077214] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo7115worker1 # [ 202.117497] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=17116worker1 # [ 202.134844] nix-daemon[835]: reaped child process 2005, status = succeeded7117worker1 # [ 202.163352] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414067118worker1 # [ 202.178640] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414067119worker1 # [ 202.193423] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.749Z level=INFO msg="All 2 paths already cached"7120worker1 # [ 202.203065] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414067121worker1 # [ 202.219542] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7122worker1 # [ 202.367285] nix-daemon[835]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)7123worker1 # [ 202.386528] nix-daemon[835]: reaped child process 2033, status = succeeded7124lb # [ 202.342082] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received uploads request method=POST path=/api/pending_closures7125worker1 # [ 202.393265] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.949Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7126worker1 # [ 202.394888] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.949Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"7127lb # [ 202.371610] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst7128worker1 # [ 202.427450] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.983Z level=INFO msg="Uploading 1 narinfos"7129lb # [ 202.388425] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7130lb # [ 202.399639] niks3-server[1152]: 2026/09/23 05:15:58 INFO Signed narinfos id=86 count=17131lb # [ 202.406305] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls7132lb # [ 202.415650] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7133lb # [ 202.421614] niks3-server[1152]: 2026/09/23 05:15:58 INFO Completed upload id=867134lb # [ 202.425106] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7135worker1 # [ 202.465446] nix-grpc-daemon[1691]: time=2026-09-23T05:15:59.019Z level=INFO msg="Upload complete. (230ms)"7136worker1 # [ 202.473099] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=7527137worker1 # [ 202.486214] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414067138worker1 # [ 202.495222] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7139worker1 # [ 202.498612] nix-daemon[835]: reaped child process 2028, status = succeeded7140worker1 # [ 202.509420] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)7141worker1 # [ 203.821942] nix-daemon[835]: reaped child process 2037, status = succeeded7142worker1 # [ 203.892886] nix-daemon[835]: accepted connection from pid 2053, user nix-grpc-daemon (trusted)7143worker1 # [ 203.912196] nix-daemon[835]: reaped child process 2056, status = succeeded7144lb # [ 203.873728] niks3-server[1152]: 2026/09/23 05:16:00 INFO Received uploads request method=POST path=/api/pending_closures7145worker1 # [ 203.927677] nix-grpc-daemon[1691]: time=2026-09-23T05:16:00.481Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7146worker1 # [ 203.929768] nix-grpc-daemon[1691]: time=2026-09-23T05:16:00.481Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7147lb # [ 203.911059] niks3-server[1152]: 2026/09/23 05:16:00 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7148worker1 # [ 203.966797] nix-grpc-daemon[1691]: time=2026-09-23T05:16:00.522Z level=INFO msg="Uploading 1 narinfos"7149lb # [ 203.925793] niks3-server[1152]: 2026/09/23 05:16:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7150lb # [ 203.937154] niks3-server[1152]: 2026/09/23 05:16:00 INFO Signed narinfos id=87 count=17151lb # [ 203.939527] niks3-server[1152]: 2026/09/23 05:16:00 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7152lb # [ 203.951055] niks3-server[1152]: 2026/09/23 05:16:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7153worker1 # [ 203.997484] nix-grpc-daemon[1691]: time=2026-09-23T05:16:00.553Z level=INFO msg="Upload complete. (274ms)"7154lb # [ 203.956572] niks3-server[1152]: 2026/09/23 05:16:00 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7155lb # [ 203.959863] niks3-server[1152]: 2026/09/23 05:16:00 INFO Completed upload id=877156lb # [ 203.961952] niks3-server[1152]: 2026/09/23 05:16:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7157worker1 # [ 204.013333] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=17158worker1 # [ 204.029033] nix-daemon[835]: reaped child process 2035, status = succeeded7159client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.43 seconds)7160worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7161worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7162client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7163client # [ 204.248813] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7164client # Running as unit: gpu.service; invocation ID: 0e084282888e44c496ca3174652c87a17165client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.08 seconds)7166lb: must succeed: curl -sf localhost:9901/clusters7167lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7168lb: must succeed: curl -sf localhost:9901/clusters7169lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7170worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7171worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7172worker1 # [ 204.447889] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414067173client # [ 204.455980] nix[1622]: don't know how to build these paths:7174client # [ 204.456993] nix[1622]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7175worker2 # [ 204.461876] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408827176worker2 # [ 204.484134] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:01Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07177client # [ 204.490524] nix[1622]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7178lb: must succeed: curl -sf localhost:9901/clusters7179lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7180lb: must succeed: curl -sf localhost:9901/clusters7181lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7182worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7183worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7184client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7185lb # [ 208.377280] envoy[900]: conn peer=[2001:db8:1::1]:50668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27183 tx=12381 ms=161627186client # [ 234.497480] nix[1622]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7187client # [ 234.512445] nix[1622]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7188client # [ 234.558053] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7189client # [ 234.566778] systemd[1]: gpu.service: Failed with result 'exit-code'.7190client # [ 234.570323] systemd[1]: gpu.service: Consumed 203ms CPU time over 30.314s wall clock time, 37.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7191client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.18 seconds)7192client: must succeed: journalctl -u gpu -o cat7193client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7194lb: must succeed: curl -sf localhost:9901/clusters7195lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7196lb: must succeed: curl -sf localhost:9901/clusters7197lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7198worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7199worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7200(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.36 seconds)7201subtest: low disk drains a worker and builds go to the other7202worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7203worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.20 seconds)7204worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7205worker1 # [ 236.361981] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:32Z level=info event=unhealthy reason=min_free7206worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)7207client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27208client # don't know how to build these paths:7209client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7210worker2 # [ 237.590419] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:408827211worker1 # [ 237.595856] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414067212worker2 # [ 237.628260] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07213worker2 # [ 237.636718] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17214worker2 # [ 237.682998] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408827215worker2 # [ 237.699797] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408827216client # copying 1 paths to 'lb:50051'...7217client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7218worker2 # [ 237.715963] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408827219worker2 # [ 237.732928] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7220worker2 # [ 237.901135] nix-daemon[858]: accepted connection from pid 2100, user nix-grpc-daemon (trusted)7221worker2 # [ 237.922576] nix-daemon[858]: reaped child process 2103, status = succeeded7222lb # [ 237.886360] niks3-server[1152]: 2026/09/23 05:16:34 INFO Received uploads request method=POST path=/api/pending_closures7223worker2 # [ 237.941062] nix-grpc-daemon[1797]: time=2026-09-23T05:16:34.504Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7224worker2 # [ 237.942836] nix-grpc-daemon[1797]: time=2026-09-23T05:16:34.504Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7225lb # [ 237.919902] niks3-server[1152]: 2026/09/23 05:16:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7226worker2 # [ 237.972822] nix-grpc-daemon[1797]: time=2026-09-23T05:16:34.536Z level=INFO msg="Uploading 1 narinfos"7227lb # [ 237.935390] niks3-server[1152]: 2026/09/23 05:16:34 INFO Signed narinfos id=88 count=17228lb # [ 237.942708] niks3-server[1152]: 2026/09/23 05:16:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7229lb # [ 237.952810] niks3-server[1152]: 2026/09/23 05:16:34 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7230lb # [ 237.967965] niks3-server[1152]: 2026/09/23 05:16:34 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7231lb # [ 237.975731] niks3-server[1152]: 2026/09/23 05:16:34 INFO Completed upload id=887232worker2 # [ 238.018212] nix-grpc-daemon[1797]: time=2026-09-23T05:16:34.582Z level=INFO msg="Upload complete. (266ms)"7233lb # [ 237.983064] niks3-server[1152]: 2026/09/23 05:16:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7234worker2 # [ 238.028322] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=5127235worker2 # [ 238.051962] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408827236worker2 # [ 238.056725] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7237worker2 # [ 238.062511] nix-daemon[858]: reaped child process 2098, status = succeeded7238client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7239client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7240worker2 # [ 238.076110] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7241lb # [ 238.387991] envoy[900]: conn peer=[2001:db8:1::1]:40962 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300807242client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7243client # farm-a> LOG-farm-a-drain7244worker2 # [ 239.563673] nix-daemon[858]: reaped child process 2107, status = succeeded7245worker2 # [ 239.657294] nix-daemon[858]: accepted connection from pid 2123, user nix-grpc-daemon (trusted)7246worker2 # [ 239.675782] nix-daemon[858]: reaped child process 2126, status = succeeded7247lb # [ 239.639420] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received uploads request method=POST path=/api/pending_closures7248worker2 # [ 239.693817] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.255Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7249worker2 # [ 239.695548] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.255Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7250lb # [ 239.680131] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7251worker2 # [ 239.734483] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.298Z level=INFO msg="Uploading 1 narinfos"7252lb # [ 239.693811] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7253lb # [ 239.702972] niks3-server[1152]: 2026/09/23 05:16:36 INFO Signed narinfos id=89 count=17254lb # [ 239.708455] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7255lb # [ 239.720301] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7256lb # [ 239.723623] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7257worker2 # [ 239.771937] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.335Z level=INFO msg="Upload complete. (284ms)"7258lb # [ 239.730225] niks3-server[1152]: 2026/09/23 05:16:36 INFO Completed upload id=897259lb # [ 239.734481] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7260worker2 # [ 239.787991] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=17261worker2 # [ 239.808214] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07262worker2 # [ 239.813511] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17263worker2 # [ 239.818973] nix-daemon[858]: reaped child process 2105, status = succeeded7264worker2 # [ 239.834673] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408827265worker2 # [ 239.850000] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408827266worker2 # [ 239.863721] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.427Z level=INFO msg="All 1 paths already cached"7267client # copying 1 paths to 'lb:50051'...7268client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7269worker2 # [ 239.874996] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408827270worker2 # [ 239.890725] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7271worker2 # [ 240.036932] nix-daemon[858]: accepted connection from pid 2132, user nix-grpc-daemon (trusted)7272worker2 # [ 240.057501] nix-daemon[858]: reaped child process 2135, status = succeeded7273lb # [ 240.013152] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received uploads request method=POST path=/api/pending_closures7274worker2 # [ 240.064206] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.627Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7275worker2 # [ 240.065660] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.627Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7276lb # [ 240.041491] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7277worker2 # [ 240.095169] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.659Z level=INFO msg="Uploading 1 narinfos"7278lb # [ 240.054930] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7279lb # [ 240.062696] niks3-server[1152]: 2026/09/23 05:16:36 INFO Signed narinfos id=90 count=17280worker2 # [ 240.110686] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.674Z level=INFO msg="Upload complete. (207ms)"7281lb # [ 240.070816] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7282lb # [ 240.077325] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7283worker2 # [ 240.119801] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=6327284lb # [ 240.080389] niks3-server[1152]: 2026/09/23 05:16:36 INFO Completed upload id=907285lb # [ 240.082617] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7286worker2 # [ 240.128858] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408827287worker2 # [ 240.134390] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7288worker2 # [ 240.138280] nix-daemon[858]: reaped child process 2130, status = succeeded7289client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7290worker2 # [ 240.149304] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7291client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7292client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7293client # farm-b> LOG-farm-b-drain7294worker2 # [ 241.832919] nix-daemon[858]: reaped child process 2139, status = succeeded7295worker2 # [ 241.941131] nix-daemon[858]: accepted connection from pid 2155, user nix-grpc-daemon (trusted)7296worker2 # [ 241.960946] nix-daemon[858]: reaped child process 2158, status = succeeded7297lb # [ 241.925955] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received uploads request method=POST path=/api/pending_closures7298worker2 # [ 241.984352] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.544Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7299worker2 # [ 241.986293] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.544Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7300lb # [ 241.970018] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7301worker2 # [ 242.023139] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.586Z level=INFO msg="Uploading 1 narinfos"7302lb # [ 241.986422] niks3-server[1152]: 2026/09/23 05:16:38 INFO Signed narinfos id=91 count=17303lb # [ 241.994301] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7304lb # [ 242.008344] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7305lb # [ 242.011924] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7306worker2 # [ 242.063191] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.627Z level=INFO msg="Upload complete. (302ms)"7307lb # [ 242.024005] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7308lb # [ 242.030057] niks3-server[1152]: 2026/09/23 05:16:38 INFO Completed upload id=917309lb # [ 242.032465] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7310worker2 # [ 242.079368] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=17311worker2 # [ 242.102326] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07312worker2 # [ 242.107797] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17313worker2 # [ 242.113254] nix-daemon[858]: reaped child process 2137, status = succeeded7314worker2 # [ 242.138923] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408827315worker2 # [ 242.154603] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:408827316worker2 # [ 242.169178] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.733Z level=INFO msg="All 2 paths already cached"7317client # copying 1 paths to 'lb:50051'...7318client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7319worker2 # [ 242.180494] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:408827320worker2 # [ 242.194348] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7321worker2 # [ 242.350961] nix-daemon[858]: accepted connection from pid 2164, user nix-grpc-daemon (trusted)7322worker2 # [ 242.371855] nix-daemon[858]: reaped child process 2167, status = succeeded7323lb # [ 242.334638] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received uploads request method=POST path=/api/pending_closures7324worker2 # [ 242.385680] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.949Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7325worker2 # [ 242.387542] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.949Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7326lb # [ 242.364663] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7327worker2 # [ 242.415293] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.979Z level=INFO msg="Uploading 1 narinfos"7328lb # [ 242.380171] niks3-server[1152]: 2026/09/23 05:16:38 INFO Signed narinfos id=92 count=17329lb # [ 242.383320] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7330lb # [ 242.393342] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7331worker2 # [ 242.447556] nix-grpc-daemon[1797]: time=2026-09-23T05:16:39.011Z level=INFO msg="Upload complete. (236ms)"7332lb # [ 242.405759] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7333lb # [ 242.414475] niks3-server[1152]: 2026/09/23 05:16:38 INFO Completed upload id=927334worker2 # [ 242.456493] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=7607335lb # [ 242.418564] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7336worker2 # [ 242.469155] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:408827337worker2 # [ 242.479401] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7338worker2 # [ 242.484697] nix-daemon[858]: reaped child process 2162, status = succeeded7339client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7340client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7341worker2 # [ 242.495514] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7342client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7343client # farm-top> LOG-farm-top-drain7344worker2 # [ 243.607535] nix-daemon[858]: reaped child process 2171, status = succeeded7345worker2 # [ 243.677332] nix-daemon[858]: accepted connection from pid 2187, user nix-grpc-daemon (trusted)7346worker2 # [ 243.693844] nix-daemon[858]: reaped child process 2190, status = succeeded7347lb # [ 243.659701] niks3-server[1152]: 2026/09/23 05:16:40 INFO Received uploads request method=POST path=/api/pending_closures7348worker2 # [ 243.714633] nix-grpc-daemon[1797]: time=2026-09-23T05:16:40.277Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7349worker2 # [ 243.716604] nix-grpc-daemon[1797]: time=2026-09-23T05:16:40.277Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7350lb # [ 243.702684] niks3-server[1152]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7351worker2 # [ 243.756348] nix-grpc-daemon[1797]: time=2026-09-23T05:16:40.320Z level=INFO msg="Uploading 1 narinfos"7352lb # [ 243.717710] niks3-server[1152]: 2026/09/23 05:16:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7353lb # [ 243.726069] niks3-server[1152]: 2026/09/23 05:16:40 INFO Signed narinfos id=93 count=17354lb # [ 243.731336] niks3-server[1152]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7355worker2 # [ 243.785935] nix-grpc-daemon[1797]: time=2026-09-23T05:16:40.349Z level=INFO msg="Upload complete. (275ms)"7356lb # [ 243.744436] niks3-server[1152]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7357lb # [ 243.749043] niks3-server[1152]: 2026/09/23 05:16:40 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7358lb # [ 243.753507] niks3-server[1152]: 2026/09/23 05:16:40 INFO Completed upload id=937359lb # [ 243.755662] niks3-server[1152]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7360worker2 # [ 243.801189] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=17361worker2 # [ 243.815989] nix-daemon[858]: reaped child process 2169, status = succeeded7362client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.52 seconds)7363client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7364client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7365worker1: must succeed: rm /nix/.rw-store/fill7366worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7367worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7368worker1 # [ 244.369556] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:40Z level=info event=healthy reason=min_free7369worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7370(finished: subtest: low disk drains a worker and builds go to the other, in 10.36 seconds)7371(finished: run the VM test script, in 246.31 seconds)7372test script finished in 246.56s7373cleanup7374kill QemuMachine (pid 49)7375client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7376client # [2026-09-23T05:16:41Z INFO virtiofsd] Client disconnected, shutting down7377client # [2026-09-23T05:16:41Z INFO virtiofsd] Client disconnected, shutting down7378client # [2026-09-23T05:16:41Z INFO virtiofsd] Client disconnected, shutting down7379kill QemuMachine (pid 50)7380lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7381lb # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7382lb # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7383lb # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7384kill QemuMachine (pid 52)7385worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7386worker1 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7387worker1 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7388worker1 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7389kill QemuMachine (pid 51)7390worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7391worker2 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7392worker2 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7393worker2 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down7394(finished: cleanup, in 1.24 seconds)7395additionally exposed symbols:7396 client, lb, worker1, worker2,7397 vlan1,7398 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh