vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #503
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1: QEMU running (pid 51)14worker1 # Formatting '/build/vm-state-worker1/tmp.AyKH72Telw', fmt=raw size=107374182415lb: QEMU running (pid 50)16worker1 # mke2fs 1.47.4 (6-Mar-2025)17client # Disk image does not exist, creating the virtualisation disk image...18client: QEMU running (pid 49)19worker2 # Disk image does not exist, creating the virtualisation disk image...20worker1 # Discarding device blocks: 0/262144 done21client # Formatting '/build/vm-state-client/tmp.Jzz2aL4Ldi', fmt=raw size=107374182422lb # Disk image does not exist, creating the virtualisation disk image...23worker2: QEMU running (pid 52)24worker2 # Formatting '/build/vm-state-worker2/tmp.ky45lD3lRa', fmt=raw size=107374182425lb # Formatting '/build/vm-state-lb/tmp.X7xj1ScF2V', fmt=raw size=107374182426worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes27client # mke2fs 1.47.4 (6-Mar-2025)28(finished: start all VMs, in 0.49 seconds)29worker1 # Filesystem UUID: 33133b99-1616-4b97-8015-09ee09300e8430client # Discarding device blocks: 0/262144 done31client: waiting for unit network-addresses-eth1.service32worker1 # Superblock backups stored on blocks:33client # Creating filesystem with 262144 4k blocks and 65536 inodes34client: waiting for the VM to finish booting35worker1 # 32768, 98304, 163840, 22937636client # Filesystem UUID: 3e628041-8142-4b90-ac7f-abf3283ad4a737lb # mke2fs 1.47.4 (6-Mar-2025)38worker2 # mke2fs 1.47.4 (6-Mar-2025)39worker1 # 40client # Superblock backups stored on blocks:41lb # Discarding device blocks: 0/262144 done42worker2 # Discarding device blocks: 0/262144 done43worker1 # Allocating group tables: 0/8 done44client # 32768, 98304, 163840, 22937645lb # Creating filesystem with 262144 4k blocks and 65536 inodes46worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes47worker1 # Writing inode tables: 0/8 done48client # 49lb # Filesystem UUID: 753fbc79-914d-4c7c-9ce8-f4b90e9bdb0a50worker2 # Filesystem UUID: afc2ae95-50d3-4000-a010-8c2d52d6604351worker1 # Creating journal (8192 blocks): done52client # Allocating group tables: 0/8 done53lb # Superblock backups stored on blocks:54worker2 # Superblock backups stored on blocks:55worker1 # Writing superblocks and filesystem accounting information: 0/8 done56client # Writing inode tables: 0/8 done57lb # 32768, 98304, 163840, 22937658worker2 # 32768, 98304, 163840, 22937659worker1 # 60client # Creating journal (8192 blocks): done61lb # 62worker2 # 63worker1 # Virtualisation disk image created.64client # Writing superblocks and filesystem accounting information: 0/8 done65lb # Allocating group tables: 0/8 done66worker2 # Allocating group tables: 0/8 done67worker1 # Starting virtiofs daemons...68client # 69lb # Writing inode tables: 0/8 done70worker2 # Writing inode tables: 0/8 done71worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72client # Virtualisation disk image created.73lb # Creating journal (8192 blocks): done74worker2 # Creating journal (8192 blocks): done75worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether76client # Starting virtiofs daemons...77lb # Writing superblocks and filesystem accounting information: 0/8 done78worker2 # Writing superblocks and filesystem accounting information: 0/8 done79worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)81lb # 82worker2 # 83worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84lb # Virtualisation disk image created.85client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether86worker2 # Virtualisation disk image created.87worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...88lb # Starting virtiofs daemons...89client # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...90worker2 # Starting virtiofs daemons...91worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...92lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)93client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)96lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether97client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether98worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether99worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether100lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)101client # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...102worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)103worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...104lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether105client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)106worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether107worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests108lb # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...109client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether110worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...111worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests112lb # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...113client # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...114worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...115worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests116lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)117client # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests118lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether119client # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests120lb # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...121client # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests122lb # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests123worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)124lb # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests125worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether126lb # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests127worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests129worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests130worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker1 # [ 0.000000] KASLR enabled134worker1 # [ 0.000000] random: crng init done135worker1 # [ 0.000000] Machine model: linux,dummy-virt136worker1 # [ 0.000000] efi: UEFI not found.137worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140worker1 # [ 0.000000] Zone ranges:141worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142worker1 # [ 0.000000] DMA32 empty143worker1 # [ 0.000000] Normal empty144worker1 # [ 0.000000] Device empty145worker1 # [ 0.000000] Movable zone start for each node146worker1 # [ 0.000000] Early memory node ranges147worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker1 # [ 0.000000] psci: probing for conduit method from DT.151worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153worker1 # [ 0.000000] psci: Trusted OS migration not required154worker1 # [ 0.000000] psci: SMC Calling Convention v1.1155worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157worker1 # [ 0.000000] Detected PIPT I-cache on CPU0158client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]159worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)160worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface161client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026162client # [ 0.000000] KASLR enabled163worker1 # [ 0.000000] CPU features: detected: Spectre-v4164client # [ 0.000000] random: crng init done165worker1 # [ 0.000000] CPU features: detected: Spectre-BHB166client # [ 0.000000] Machine model: linux,dummy-virt167worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38168client # [ 0.000000] efi: UEFI not found.169worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23170client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT171worker1 # [ 0.000000] alternatives: applying boot alternatives172client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]173client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]174client # [ 0.000000] Zone ranges:175client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]176client # [ 0.000000] DMA32 empty177client # [ 0.000000] Normal empty178client # [ 0.000000] Device empty179client # [ 0.000000] Movable zone start for each node180worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nkxfr43rzbz80xc8q1a00w66rjj2an3s-nixos-system-worker1-test/init regInfo=/nix/store/s2vfldvxw21afqph8awka7bg9gka3b50-closure-info/registration console=ttyAMA0,115200n8 console=tty0181client # [ 0.000000] Early memory node ranges182worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]183client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]184client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]185worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/s2vfldvxw21afqph8awka7bg9gka3b50-closure-info/registration", will be passed to user space.186client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000187worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026188worker2 # [ 0.000000] KASLR enabled189lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]190client # [ 0.000000] psci: probing for conduit method from DT.191worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes192worker2 # [ 0.000000] random: crng init done193client # [ 0.000000] psci: PSCIv1.3 detected in firmware.194worker2 # [ 0.000000] Machine model: linux,dummy-virt195worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)196client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs197worker2 # [ 0.000000] efi: UEFI not found.198lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026199client # [ 0.000000] psci: Trusted OS migration not required200lb # [ 0.000000] KASLR enabled201worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)202worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT203lb # [ 0.000000] random: crng init done204client # [ 0.000000] psci: SMC Calling Convention v1.1205lb # [ 0.000000] Machine model: linux,dummy-virt206worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB207worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]208lb # [ 0.000000] efi: UEFI not found.209client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)210worker1 # [ 0.000000] software IO TLB: area num 1.211worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]212lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT213client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296214worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB215worker2 # [ 0.000000] Zone ranges:216client # [ 0.000000] Detected PIPT I-cache on CPU0217worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]218lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]219worker2 # [ 0.000000] DMA32 empty220worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)221worker2 # [ 0.000000] Normal empty222worker1 # [ 0.000000] Fallback order for Node 0: 0223lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]224worker2 # [ 0.000000] Device empty225lb # [ 0.000000] Zone ranges:226worker2 # [ 0.000000] Movable zone start for each node227worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216228lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]229worker2 # [ 0.000000] Early memory node ranges230worker1 # [ 0.000000] Policy zone: DMA231lb # [ 0.000000] DMA32 empty232client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)233lb # [ 0.000000] Normal empty234worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off235worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]236lb # [ 0.000000] Device empty237lb # [ 0.000000] Movable zone start for each node238worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1239worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]240lb # [ 0.000000] Early memory node ranges241worker1 # [ 0.000000] allocated 3145728 bytes of page_ext242worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000243client # [ 0.000000] CPU features: detected: GICv3 CPU interface244lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]245worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages246worker2 # [ 0.000000] psci: probing for conduit method from DT.247client # [ 0.000000] CPU features: detected: Spectre-v4248worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups249lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]250worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.251client # [ 0.000000] CPU features: detected: Spectre-BHB252worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.253lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000254worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs255client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38256worker1 # [ 0.000000] rcu: RCU event tracing is enabled.257worker2 # [ 0.000000] psci: Trusted OS migration not required258lb # [ 0.000000] psci: probing for conduit method from DT.259client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23260worker2 # [ 0.000000] psci: SMC Calling Convention v1.1261worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.262lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.263client # [ 0.000000] alternatives: applying boot alternatives264lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs265worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.266worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)267worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.268lb # [ 0.000000] psci: Trusted OS migration not required269lb # [ 0.000000] psci: SMC Calling Convention v1.1270worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.271worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296272worker2 # [ 0.000000] Detected PIPT I-cache on CPU0273worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.274lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)275worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)276worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1277lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296278worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface279lb # [ 0.000000] Detected PIPT I-cache on CPU0280client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test/init regInfo=/nix/store/bvnhgdmv5qvfh2jgky9d95hczimkx039-closure-info/registration console=ttyAMA0,115200n8 console=tty0281worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.282worker2 # [ 0.000000] CPU features: detected: Spectre-v4283worker2 # [ 0.000000] CPU features: detected: Spectre-BHB284lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)285worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.286worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38287lb # [ 0.000000] CPU features: detected: GICv3 CPU interface288lb # [ 0.000000] CPU features: detected: Spectre-v4289worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23290worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.291lb # [ 0.000000] CPU features: detected: Spectre-BHB292client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bvnhgdmv5qvfh2jgky9d95hczimkx039-closure-info/registration", will be passed to user space.293worker2 # [ 0.000000] alternatives: applying boot alternatives294worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0295lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38296worker1 # [ 0.000000] GICv3: 256 SPIs implemented297client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes298lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23299worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented300lb # [ 0.000000] alternatives: applying boot alternatives301client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)302worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq303worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI304client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)305worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0306client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB307worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/91x7j7xpp6q86mih2akg81hmd36hcvqs-nixos-system-worker2-test/init regInfo=/nix/store/i3k5wlp1fqfgi1004a4pn264iky91pbm-closure-info/registration console=ttyAMA0,115200n8 console=tty0308worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000309client # [ 0.000000] software IO TLB: area num 1.310worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]311client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB312lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test/init regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration console=ttyAMA0,115200n8 console=tty0313worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)314client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)315worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i3k5wlp1fqfgi1004a4pn264iky91pbm-closure-info/registration", will be passed to user space.316client # [ 0.000000] Fallback order for Node 0: 0317worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)318client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216319worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes320client # [ 0.000000] Policy zone: DMA321worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000322lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration", will be passed to user space.323worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)324client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off325worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000326lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes327worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)328client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1329worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.330client # [ 0.000000] allocated 3145728 bytes of page_ext331lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)332worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB333client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages334worker2 # [ 0.000000] software IO TLB: area num 1.335worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns336lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)337client # [ 0.000000] ftrace: allocated 294 pages with 4 groups338worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB339worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).340client # [ 0.000000] rcu: Hierarchical RCU implementation.341lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB342lb # [ 0.000000] software IO TLB: area num 1.343client # [ 0.000000] rcu: RCU event tracing is enabled.344worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)345worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns346worker2 # [ 0.000000] Fallback order for Node 0: 0347lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB348client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.349worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216350worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns351client # [ 0.000000] Trampoline variant of Tasks RCU enabled.352worker2 # [ 0.000000] Policy zone: DMA353lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)354worker1 # [ 0.000034] arm-pv: using stolen time PV355client # [ 0.000000] Rude variant of Tasks RCU enabled.356lb # [ 0.000000] Fallback order for Node 0: 0357worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off358client # [ 0.000000] Tracing variant of Tasks RCU enabled.359lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216360worker1 # [ 0.000658] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)361lb # [ 0.000000] Policy zone: DMA362client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.363worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1364worker1 # [ 0.000822] Console: colour dummy device 80x25365lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off366worker2 # [ 0.000000] allocated 3145728 bytes of page_ext367client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1368worker1 # [ 0.000830] printk: legacy console [tty0] enabled369worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages370lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1371client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.372worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups373lb # [ 0.000000] allocated 3145728 bytes of page_ext374worker1 # [ 0.001030] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)375worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.376lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages377worker1 # [ 0.001036] pid_max: default: 32768 minimum: 301378client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.379worker2 # [ 0.000000] rcu: RCU event tracing is enabled.380lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups381worker1 # [ 0.001110] LSM: initializing lsm=capability,landlock,yama,bpf,ima382worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.383lb # [ 0.000000] rcu: Hierarchical RCU implementation.384worker1 # [ 0.001292] landlock: Up and running.385client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.386worker1 # [ 0.001295] Yama: becoming mindful.387lb # [ 0.000000] rcu: RCU event tracing is enabled.388worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.389client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0390worker1 # [ 0.001982] LSM support for eBPF active391worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.392lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.393client # [ 0.000000] GICv3: 256 SPIs implemented394worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.395worker1 # [ 0.002153] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)396lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.397client # [ 0.000000] GICv3: 0 Extended SPIs implemented398lb # [ 0.000000] Rude variant of Tasks RCU enabled.399client # [ 0.000000] Root IRQ handler: gic_handle_irq400worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.401worker1 # [ 0.002185] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)402lb # [ 0.000000] Tracing variant of Tasks RCU enabled.403client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI404worker1 # [ 0.004102] rcu: Hierarchical SRCU implementation.405worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1406client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0407worker1 # [ 0.004107] rcu: Max phase no-delay instances is 1000.408lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.409client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000410worker1 # [ 0.005376] fsl-mc MSI: its@8080000 domain created411worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.412lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1413client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]414worker1 # [ 0.005464] EFI services will not be available.415worker1 # [ 0.005532] smp: Bringing up secondary CPUs ...416worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.417lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.418client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)419worker1 # [ 0.005540] smp: Brought up 1 node, 1 CPU420worker1 # [ 0.005543] SMP: Total of 1 processors activated.421lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.422worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.423worker1 # [ 0.005548] CPU: All CPU(s) started at EL1424client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)425worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0426lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.427worker1 # [ 0.005558] CPU features: detected: Branch Target Identification428client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000429worker2 # [ 0.000000] GICv3: 256 SPIs implemented430lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0431worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented432worker1 # [ 0.005562] CPU features: detected: ARMv8.4 Translation Table Level433client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000434lb # [ 0.000000] GICv3: 256 SPIs implemented435worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq436lb # [ 0.000000] GICv3: 0 Extended SPIs implemented437worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI438worker1 # [ 0.005565] CPU features: detected: Instruction cache invalidation not required for I/D coherence439client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.440lb # [ 0.000000] Root IRQ handler: gic_handle_irq441worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0442lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI443worker1 # [ 0.005569] CPU features: detected: Data cache clean to the PoU not required for I/D coherence444client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns445worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000446lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0447worker1 # [ 0.005573] CPU features: detected: Common not Private translations448client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).449worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]450lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000451worker1 # [ 0.005576] CPU features: detected: CRC32 instructions452lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]453worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)454client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns455worker1 # [ 0.005579] CPU features: detected: Data cache clean to Point of Deep Persistence456lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)457worker1 # [ 0.005582] CPU features: detected: Data cache clean to Point of Persistence458client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns459worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)460client # [ 0.000033] arm-pv: using stolen time PV461worker1 # [ 0.005586] CPU features: detected: Data independent timing control (DIT)462worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000463lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)464worker1 # [ 0.005589] CPU features: detected: E0PD465worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000466client # [ 0.000537] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)467lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000468worker1 # [ 0.005591] CPU features: detected: Enhanced Counter Virtualization469client # [ 0.000943] Console: colour dummy device 80x25470worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.471lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000472client # [ 0.000950] printk: legacy console [tty0] enabled473worker1 # [ 0.005594] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)474lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.475worker1 # [ 0.005598] CPU features: detected: Enhanced Virtualization Traps476worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns477client # [ 0.001142] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)478worker1 # [ 0.005601] CPU features: detected: Fine Grained Traps479worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).480client # [ 0.001148] pid_max: default: 32768 minimum: 301481lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns482worker1 # [ 0.005604] CPU features: detected: Generic authentication (architected QARMA5 algorithm)483client # [ 0.001224] LSM: initializing lsm=capability,landlock,yama,bpf,ima484lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).485worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns486client # [ 0.001358] landlock: Up and running.487worker1 # [ 0.005609] CPU features: detected: RCpc load-acquire (LDAPR)488client # [ 0.001360] Yama: becoming mindful.489lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns490worker1 # [ 0.005611] CPU features: detected: LSE atomic instructions491worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns492client # [ 0.001833] LSM support for eBPF active493worker2 # [ 0.000029] arm-pv: using stolen time PV494worker1 # [ 0.005614] CPU features: detected: Privileged Access Never495client # [ 0.001980] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)496lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns497worker1 # [ 0.005617] CPU features: detected: PMUv3498lb # [ 0.000031] arm-pv: using stolen time PV499worker1 # [ 0.005620] CPU features: detected: RAS Extension Support500worker2 # [ 0.000388] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)501client # [ 0.002013] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)502worker2 # [ 0.000556] Console: colour dummy device 80x25503client # [ 0.003836] rcu: Hierarchical SRCU implementation.504worker1 # [ 0.005622] CPU features: detected: RASv1p1 Extension Support505lb # [ 0.000405] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)506worker2 # [ 0.000563] printk: legacy console [tty0] enabled507client # [ 0.003841] rcu: Max phase no-delay instances is 1000.508worker1 # [ 0.005625] CPU features: detected: Random Number Generator509lb # [ 0.000574] Console: colour dummy device 80x25510client # [ 0.005104] fsl-mc MSI: its@8080000 domain created511lb # [ 0.000581] printk: legacy console [tty0] enabled512worker1 # [ 0.005627] CPU features: detected: Speculation barrier (SB)513client # [ 0.005194] EFI services will not be available.514worker2 # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)515client # [ 0.005274] smp: Bringing up secondary CPUs ...516worker2 # [ 0.000782] pid_max: default: 32768 minimum: 301517worker1 # [ 0.005630] CPU features: detected: Stage-2 Force Write-Back518lb # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)519client # [ 0.005283] smp: Brought up 1 node, 1 CPU520worker2 # [ 0.000864] LSM: initializing lsm=capability,landlock,yama,bpf,ima521worker1 # [ 0.005633] CPU features: detected: TLB range maintenance instructions522lb # [ 0.000782] pid_max: default: 32768 minimum: 301523client # [ 0.005286] SMP: Total of 1 processors activated.524worker2 # [ 0.000992] landlock: Up and running.525client # [ 0.005291] CPU: All CPU(s) started at EL1526worker1 # [ 0.005637] CPU features: detected: Speculative Store Bypassing Safe (SSBS)527worker2 # [ 0.000996] Yama: becoming mindful.528lb # [ 0.000863] LSM: initializing lsm=capability,landlock,yama,bpf,ima529lb # [ 0.000998] landlock: Up and running.530worker2 # [ 0.001422] LSM support for eBPF active531client # [ 0.005302] CPU features: detected: Branch Target Identification532worker1 # [ 0.005677] alternatives: applying system-wide alternatives533lb # [ 0.001000] Yama: becoming mindful.534lb # [ 0.001434] LSM support for eBPF active535client # [ 0.005307] CPU features: detected: ARMv8.4 Translation Table Level536worker2 # [ 0.001542] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)537worker1 # [ 0.008657] CPU features: detected: BBM Level 2 without TLB conflict abort538lb # [ 0.001565] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)539worker2 # [ 0.001576] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)540client # [ 0.005310] CPU features: detected: Instruction cache invalidation not required for I/D coherence541worker2 # [ 0.003585] rcu: Hierarchical SRCU implementation.542lb # [ 0.001598] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)543worker1 # [ 0.008847] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)544client # [ 0.005314] CPU features: detected: Data cache clean to the PoU not required for I/D coherence545worker2 # [ 0.003590] rcu: Max phase no-delay instances is 1000.546worker1 # [ 0.009252] devtmpfs: initialized547lb # [ 0.003450] rcu: Hierarchical SRCU implementation.548worker2 # [ 0.004860] fsl-mc MSI: its@8080000 domain created549client # [ 0.005318] CPU features: detected: Common not Private translations550lb # [ 0.003455] rcu: Max phase no-delay instances is 1000.551worker1 # [ 0.010992] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)552worker2 # [ 0.004952] EFI services will not be available.553client # [ 0.005321] CPU features: detected: CRC32 instructions554lb # [ 0.004709] fsl-mc MSI: its@8080000 domain created555worker2 # [ 0.005084] smp: Bringing up secondary CPUs ...556worker1 # [ 0.011014] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).557lb # [ 0.004805] EFI services will not be available.558worker2 # [ 0.005093] smp: Brought up 1 node, 1 CPU559client # [ 0.005324] CPU features: detected: Data cache clean to Point of Deep Persistence560lb # [ 0.004879] smp: Bringing up secondary CPUs ...561worker2 # [ 0.005096] SMP: Total of 1 processors activated.562worker1 # [ 0.011185] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL563client # [ 0.005327] CPU features: detected: Data cache clean to Point of Persistence564lb # [ 0.004888] smp: Brought up 1 node, 1 CPU565worker2 # [ 0.005102] CPU: All CPU(s) started at EL1566lb # [ 0.004891] SMP: Total of 1 processors activated.567client # [ 0.005330] CPU features: detected: Data independent timing control (DIT)568worker2 # [ 0.005113] CPU features: detected: Branch Target Identification569lb # [ 0.004896] CPU: All CPU(s) started at EL1570client # [ 0.005333] CPU features: detected: E0PD571worker2 # [ 0.005118] CPU features: detected: ARMv8.4 Translation Table Level572lb # [ 0.004907] CPU features: detected: Branch Target Identification573client # [ 0.005336] CPU features: detected: Enhanced Counter Virtualization574lb # [ 0.004911] CPU features: detected: ARMv8.4 Translation Table Level575worker2 # [ 0.005121] CPU features: detected: Instruction cache invalidation not required for I/D coherence576client # [ 0.005339] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)577client # [ 0.005343] CPU features: detected: Enhanced Virtualization Traps578lb # [ 0.004914] CPU features: detected: Instruction cache invalidation not required for I/D coherence579worker2 # [ 0.005125] CPU features: detected: Data cache clean to the PoU not required for I/D coherence580client # [ 0.005345] CPU features: detected: Fine Grained Traps581worker2 # [ 0.005129] CPU features: detected: Common not Private translations582lb # [ 0.004918] CPU features: detected: Data cache clean to the PoU not required for I/D coherence583client # [ 0.005349] CPU features: detected: Generic authentication (architected QARMA5 algorithm)584worker2 # [ 0.005132] CPU features: detected: CRC32 instructions585lb # [ 0.004922] CPU features: detected: Common not Private translations586client # [ 0.005354] CPU features: detected: RCpc load-acquire (LDAPR)587lb # [ 0.004925] CPU features: detected: CRC32 instructions588worker2 # [ 0.005135] CPU features: detected: Data cache clean to Point of Deep Persistence589client # [ 0.005357] CPU features: detected: LSE atomic instructions590lb # [ 0.004928] CPU features: detected: Data cache clean to Point of Deep Persistence591worker2 # [ 0.005138] CPU features: detected: Data cache clean to Point of Persistence592client # [ 0.005360] CPU features: detected: Privileged Access Never593client # [ 0.005362] CPU features: detected: PMUv3594lb # [ 0.004932] CPU features: detected: Data cache clean to Point of Persistence595worker2 # [ 0.005141] CPU features: detected: Data independent timing control (DIT)596client # [ 0.005365] CPU features: detected: RAS Extension Support597worker2 # [ 0.005144] CPU features: detected: E0PD598worker1 # [ 0.011189] 0 pages in range for non-PLT usage599lb # [ 0.004935] CPU features: detected: Data independent timing control (DIT)600client # [ 0.005368] CPU features: detected: RASv1p1 Extension Support601lb # [ 0.004938] CPU features: detected: E0PD602worker1 # [ 0.011190] 508272 pages in range for PLT usage603worker2 # [ 0.005147] CPU features: detected: Enhanced Counter Virtualization604client # [ 0.005371] CPU features: detected: Random Number Generator605worker1 # [ 0.011284] pinctrl core: initialized pinctrl subsystem606lb # [ 0.004940] CPU features: detected: Enhanced Counter Virtualization607worker2 # [ 0.005150] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)608worker1 # [ 0.012058] DMI not present or invalid.609client # [ 0.005373] CPU features: detected: Speculation barrier (SB)610lb # [ 0.004944] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)611worker2 # [ 0.005153] CPU features: detected: Enhanced Virtualization Traps612worker1 # [ 0.015395] NET: Registered PF_NETLINK/PF_ROUTE protocol family613client # [ 0.005376] CPU features: detected: Stage-2 Force Write-Back614lb # [ 0.004947] CPU features: detected: Enhanced Virtualization Traps615worker2 # [ 0.005156] CPU features: detected: Fine Grained Traps616worker1 # [ 0.017852] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations617client # [ 0.005379] CPU features: detected: TLB range maintenance instructions618lb # [ 0.004950] CPU features: detected: Fine Grained Traps619worker2 # [ 0.005160] CPU features: detected: Generic authentication (architected QARMA5 algorithm)620client # [ 0.005384] CPU features: detected: Speculative Store Bypassing Safe (SSBS)621worker1 # [ 0.018042] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations622lb # [ 0.004954] CPU features: detected: Generic authentication (architected QARMA5 algorithm)623worker2 # [ 0.005165] CPU features: detected: RCpc load-acquire (LDAPR)624client # [ 0.005420] alternatives: applying system-wide alternatives625worker1 # [ 0.018249] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations626lb # [ 0.004959] CPU features: detected: RCpc load-acquire (LDAPR)627worker2 # [ 0.005168] CPU features: detected: LSE atomic instructions628client # [ 0.008387] CPU features: detected: BBM Level 2 without TLB conflict abort629lb # [ 0.004962] CPU features: detected: LSE atomic instructions630worker1 # [ 0.018272] audit: initializing netlink subsys (disabled)631worker2 # [ 0.005171] CPU features: detected: Privileged Access Never632lb # [ 0.004965] CPU features: detected: Privileged Access Never633worker1 # [ 0.018807] thermal_sys: Registered thermal governor 'fair_share'634worker2 # [ 0.005174] CPU features: detected: PMUv3635lb # [ 0.004968] CPU features: detected: PMUv3636worker2 # [ 0.005176] CPU features: detected: RAS Extension Support637worker1 # [ 0.018809] thermal_sys: Registered thermal governor 'bang_bang'638client # [ 0.008578] Memory: 1405632K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)639lb # [ 0.004971] CPU features: detected: RAS Extension Support640client # [ 0.009023] devtmpfs: initialized641worker2 # [ 0.005179] CPU features: detected: RASv1p1 Extension Support642worker1 # [ 0.018813] thermal_sys: Registered thermal governor 'step_wise'643lb # [ 0.004973] CPU features: detected: RASv1p1 Extension Support644client # [ 0.010850] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)645worker2 # [ 0.005182] CPU features: detected: Random Number Generator646worker1 # [ 0.018816] thermal_sys: Registered thermal governor 'user_space'647lb # [ 0.004976] CPU features: detected: Random Number Generator648worker2 # [ 0.005184] CPU features: detected: Speculation barrier (SB)649worker1 # [ 0.018821] thermal_sys: Registered thermal governor 'power_allocator'650client # [ 0.010873] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).651lb # [ 0.004979] CPU features: detected: Speculation barrier (SB)652worker2 # [ 0.005187] CPU features: detected: Stage-2 Force Write-Back653client # [ 0.011073] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL654worker1 # [ 0.018847] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1655lb # [ 0.004981] CPU features: detected: Stage-2 Force Write-Back656worker2 # [ 0.005190] CPU features: detected: TLB range maintenance instructions657worker1 # [ 0.018855] cpuidle: using governor ladder658client # [ 0.011077] 0 pages in range for non-PLT usage659lb # [ 0.004984] CPU features: detected: TLB range maintenance instructions660worker1 # [ 0.018861] cpuidle: using governor menu661client # [ 0.011079] 508272 pages in range for PLT usage662worker2 # [ 0.005194] CPU features: detected: Speculative Store Bypassing Safe (SSBS)663lb # [ 0.004989] CPU features: detected: Speculative Store Bypassing Safe (SSBS)664worker1 # [ 0.019054] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.665client # [ 0.011178] pinctrl core: initialized pinctrl subsystem666worker2 # [ 0.005232] alternatives: applying system-wide alternatives667client # [ 0.011995] DMI not present or invalid.668lb # [ 0.005026] alternatives: applying system-wide alternatives669worker1 # [ 0.019071] ASID allocator initialised with 65536 entries670worker2 # [ 0.008191] CPU features: detected: BBM Level 2 without TLB conflict abort671client # [ 0.015089] NET: Registered PF_NETLINK/PF_ROUTE protocol family672worker1 # [ 0.020307] Serial: AMBA PL011 UART driver673lb # [ 0.008067] CPU features: detected: BBM Level 2 without TLB conflict abort674client # [ 0.017308] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations675worker1 # [ 0.025443] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1676worker2 # [ 0.008372] Memory: 1405688K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)677worker1 # [ 0.025595] printk: console [ttyAMA0] enabled678client # [ 0.017499] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations679worker2 # [ 0.008794] devtmpfs: initialized680lb # [ 0.008316] Memory: 1405644K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)681lb # [ 0.008681] devtmpfs: initialized682worker2 # [ 0.010606] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)683client # [ 0.017696] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations684worker1 # [ 0.146502] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages685lb # [ 0.010491] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)686client # [ 0.017718] audit: initializing netlink subsys (disabled)687worker1 # [ 0.146522] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page688worker2 # [ 0.010628] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).689lb # [ 0.010513] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).690client # [ 0.018263] thermal_sys: Registered thermal governor 'fair_share'691worker1 # [ 0.146527] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages692client # [ 0.018265] thermal_sys: Registered thermal governor 'bang_bang'693lb # [ 0.010693] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL694worker1 # [ 0.146531] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page695client # [ 0.018268] thermal_sys: Registered thermal governor 'step_wise'696worker1 # [ 0.146536] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages697client # [ 0.018271] thermal_sys: Registered thermal governor 'user_space'698worker1 # [ 0.146540] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page699worker1 # [ 0.146544] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages700worker1 # [ 0.146549] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page701worker1 # [ 0.154052] fbcon: Taking over console702worker1 # [ 0.154066] ACPI: Interpreter disabled.703worker1 # [ 0.155928] iommu: Default domain type: Translated704worker2 # [ 0.010802] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL705worker1 # [ 0.155938] iommu: DMA domain TLB invalidation policy: strict mode706worker2 # [ 0.010807] 0 pages in range for non-PLT usage707lb # [ 0.010697] 0 pages in range for non-PLT usage708worker1 # [ 0.157660] SCSI subsystem initialized709worker2 # [ 0.010808] 508272 pages in range for PLT usage710lb # [ 0.010699] 508272 pages in range for PLT usage711worker2 # [ 0.010920] pinctrl core: initialized pinctrl subsystem712lb # [ 0.010792] pinctrl core: initialized pinctrl subsystem713worker2 # [ 0.011658] DMI not present or invalid.714lb # [ 0.011567] DMI not present or invalid.715client # [ 0.018276] thermal_sys: Registered thermal governor 'power_allocator'716worker2 # [ 0.014721] NET: Registered PF_NETLINK/PF_ROUTE protocol family717lb # [ 0.014812] NET: Registered PF_NETLINK/PF_ROUTE protocol family718client # [ 0.018301] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1719worker2 # [ 0.016986] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations720client # [ 0.018310] cpuidle: using governor ladder721lb # [ 0.017002] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations722client # [ 0.018316] cpuidle: using governor menu723worker2 # [ 0.017167] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations724lb # [ 0.017185] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations725client # [ 0.018492] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.726worker2 # [ 0.017366] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations727lb # [ 0.017383] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations728client # [ 0.018507] ASID allocator initialised with 65536 entries729worker2 # [ 0.017389] audit: initializing netlink subsys (disabled)730client # [ 0.019702] Serial: AMBA PL011 UART driver731lb # [ 0.017405] audit: initializing netlink subsys (disabled)732worker2 # [ 0.017975] thermal_sys: Registered thermal governor 'fair_share'733lb # [ 0.017948] thermal_sys: Registered thermal governor 'fair_share'734client # [ 0.024902] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1735worker2 # [ 0.017977] thermal_sys: Registered thermal governor 'bang_bang'736lb # [ 0.017950] thermal_sys: Registered thermal governor 'bang_bang'737client # [ 0.025061] printk: console [ttyAMA0] enabled738worker2 # [ 0.017981] thermal_sys: Registered thermal governor 'step_wise'739lb # [ 0.017954] thermal_sys: Registered thermal governor 'step_wise'740worker2 # [ 0.017984] thermal_sys: Registered thermal governor 'user_space'741lb # [ 0.017956] thermal_sys: Registered thermal governor 'user_space'742client # [ 0.148049] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages743worker1 # [ 0.165465] usbcore: registered new interface driver usbfs744worker2 # [ 0.017989] thermal_sys: Registered thermal governor 'power_allocator'745client # [ 0.148069] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page746lb # [ 0.017963] thermal_sys: Registered thermal governor 'power_allocator'747worker1 # [ 0.165490] usbcore: registered new interface driver hub748worker2 # [ 0.018015] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1749worker1 # [ 0.165514] usbcore: registered new device driver usb750client # [ 0.148074] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages751lb # [ 0.017988] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1752worker2 # [ 0.018023] cpuidle: using governor ladder753worker1 # [ 0.165781] pps_core: LinuxPPS API ver. 1 registered754lb # [ 0.017997] cpuidle: using governor ladder755client # [ 0.148078] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page756worker2 # [ 0.018028] cpuidle: using governor menu757lb # [ 0.018003] cpuidle: using governor menu758worker1 # [ 0.165787] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>759client # [ 0.148082] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages760worker2 # [ 0.018209] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.761lb # [ 0.018184] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.762worker1 # [ 0.165797] PTP clock support registered763client # [ 0.148087] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page764worker2 # [ 0.018225] ASID allocator initialised with 65536 entries765worker1 # [ 0.165841] EDAC MC: Ver: 3.0.0766lb # [ 0.018199] ASID allocator initialised with 65536 entries767worker2 # [ 0.019383] Serial: AMBA PL011 UART driver768lb # [ 0.019419] Serial: AMBA PL011 UART driver769client # [ 0.148091] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages770worker1 # [ 0.170329] scmi_core: SCMI protocol bus registered771worker2 # [ 0.024646] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1772client # [ 0.148096] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page773lb # [ 0.024684] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1774worker2 # [ 0.024773] printk: console [ttyAMA0] enabled775worker1 # [ 0.171267] FPGA manager framework776lb # [ 0.024838] printk: console [ttyAMA0] enabled777worker1 # [ 0.172266] vgaarb: loaded778worker2 # [ 0.145448] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages779lb # [ 0.143799] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages780worker1 # [ 0.172899] clocksource: Switched to clocksource arch_sys_counter781client # [ 0.155529] fbcon: Taking over console782worker2 # [ 0.145473] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page783lb # [ 0.143819] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page784client # [ 0.155544] ACPI: Interpreter disabled.785worker1 # [ 0.174260] VFS: Disk quotas dquot_6.6.0786worker2 # [ 0.145479] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages787client # [ 0.157401] iommu: Default domain type: Translated788lb # [ 0.143824] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages789worker1 # [ 0.174301] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)790worker2 # [ 0.145483] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page791lb # [ 0.143828] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page792client # [ 0.157411] iommu: DMA domain TLB invalidation policy: strict mode793worker1 # [ 0.175862] netfs: FS-Cache loaded794worker2 # [ 0.145488] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages795worker1 # [ 0.175983] pnp: PnP ACPI: disabled796lb # [ 0.143833] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages797client # [ 0.159096] SCSI subsystem initialized798worker2 # [ 0.145493] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page799lb # [ 0.143837] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page800worker2 # [ 0.145497] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages801lb # [ 0.143842] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages802worker2 # [ 0.145501] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page803lb # [ 0.143846] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page804worker2 # [ 0.152979] fbcon: Taking over console805lb # [ 0.151173] fbcon: Taking over console806worker2 # [ 0.152997] ACPI: Interpreter disabled.807lb # [ 0.151186] ACPI: Interpreter disabled.808worker2 # [ 0.154920] iommu: Default domain type: Translated809lb # [ 0.153048] iommu: Default domain type: Translated810worker2 # [ 0.154932] iommu: DMA domain TLB invalidation policy: strict mode811lb # [ 0.153057] iommu: DMA domain TLB invalidation policy: strict mode812lb # [ 0.154782] SCSI subsystem initialized813worker2 # [ 0.156772] SCSI subsystem initialized814client # [ 0.166936] usbcore: registered new interface driver usbfs815worker1 # [ 0.181820] NET: Registered PF_INET protocol family816client # [ 0.166960] usbcore: registered new interface driver hub817worker1 # [ 0.182090] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)818client # [ 0.166985] usbcore: registered new device driver usb819client # [ 0.167305] pps_core: LinuxPPS API ver. 1 registered820client # [ 0.167311] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>821client # [ 0.167321] PTP clock support registered822client # [ 0.167368] EDAC MC: Ver: 3.0.0823client # [ 0.172080] scmi_core: SCMI protocol bus registered824client # [ 0.173046] FPGA manager framework825client # [ 0.173969] vgaarb: loaded826worker2 # [ 0.163846] usbcore: registered new interface driver usbfs827worker2 # [ 0.163882] usbcore: registered new interface driver hub828lb # [ 0.162616] usbcore: registered new interface driver usbfs829client # [ 0.174639] clocksource: Switched to clocksource arch_sys_counter830worker2 # [ 0.163899] usbcore: registered new device driver usb831lb # [ 0.162642] usbcore: registered new interface driver hub832worker2 # [ 0.164200] pps_core: LinuxPPS API ver. 1 registered833client # [ 0.176083] VFS: Disk quotas dquot_6.6.0834lb # [ 0.162666] usbcore: registered new device driver usb835lb # [ 0.162952] pps_core: LinuxPPS API ver. 1 registered836client # [ 0.176124] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)837worker2 # [ 0.164207] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>838worker2 # [ 0.164218] PTP clock support registered839client # [ 0.177661] netfs: FS-Cache loaded840lb # [ 0.162958] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>841worker2 # [ 0.164271] EDAC MC: Ver: 3.0.0842client # [ 0.177760] pnp: PnP ACPI: disabled843lb # [ 0.162969] PTP clock support registered844lb # [ 0.163009] EDAC MC: Ver: 3.0.0845worker2 # [ 0.168903] scmi_core: SCMI protocol bus registered846lb # [ 0.167581] scmi_core: SCMI protocol bus registered847worker2 # [ 0.169882] FPGA manager framework848worker2 # [ 0.170869] vgaarb: loaded849lb # [ 0.168544] FPGA manager framework850lb # [ 0.169531] vgaarb: loaded851worker2 # [ 0.171517] clocksource: Switched to clocksource arch_sys_counter852lb # [ 0.170165] clocksource: Switched to clocksource arch_sys_counter853lb # [ 0.171542] VFS: Disk quotas dquot_6.6.0854worker2 # [ 0.173829] VFS: Disk quotas dquot_6.6.0855lb # [ 0.171573] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)856worker2 # [ 0.173864] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)857lb # [ 0.173126] netfs: FS-Cache loaded858lb # [ 0.173240] pnp: PnP ACPI: disabled859client # [ 0.183825] NET: Registered PF_INET protocol family860worker2 # [ 0.177604] netfs: FS-Cache loaded861client # [ 0.184087] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)862worker2 # [ 0.177738] pnp: PnP ACPI: disabled863worker2 # [ 0.181799] NET: Registered PF_INET protocol family864lb # [ 0.179107] NET: Registered PF_INET protocol family865worker2 # [ 0.182064] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)866lb # [ 0.179376] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)867worker1 # [ 0.211559] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)868worker1 # [ 0.211603] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)869worker1 # [ 0.211632] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)870worker1 # [ 0.211703] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)871worker1 # [ 0.211845] TCP: Hash tables configured (established 16384 bind 16384)872worker1 # [ 0.211927] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)873worker1 # [ 0.211962] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)874worker1 # [ 0.211998] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)875worker1 # [ 0.212087] NET: Registered PF_UNIX/PF_LOCAL protocol family876worker1 # [ 0.212107] NET: Registered PF_XDP protocol family877worker1 # [ 0.212125] PCI: CLS 0 bytes, default 64878worker1 # [ 0.212361] Trying to unpack rootfs image as initramfs...879worker1 # [ 0.230441] kvm [1]: HYP mode not available880client # [ 0.213660] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881client # [ 0.213704] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882client # [ 0.213733] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883client # [ 0.213806] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884client # [ 0.213950] TCP: Hash tables configured (established 16384 bind 16384)885client # [ 0.214038] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)886client # [ 0.214074] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)887client # [ 0.214126] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)888worker2 # [ 0.211982] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)889client # [ 0.214221] NET: Registered PF_UNIX/PF_LOCAL protocol family890lb # [ 0.209444] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)891client # [ 0.214242] NET: Registered PF_XDP protocol family892worker2 # [ 0.212034] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)893client # [ 0.214261] PCI: CLS 0 bytes, default 64894lb # [ 0.209492] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)895client # [ 0.214516] Trying to unpack rootfs image as initramfs...896worker2 # [ 0.212071] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)897lb # [ 0.209520] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)898worker2 # [ 0.212151] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)899lb # [ 0.209593] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)900worker2 # [ 0.212299] TCP: Hash tables configured (established 16384 bind 16384)901lb # [ 0.209736] TCP: Hash tables configured (established 16384 bind 16384)902worker2 # [ 0.212414] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)903lb # [ 0.209823] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)904worker2 # [ 0.212467] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)905lb # [ 0.209861] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)906worker2 # [ 0.212505] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)907lb # [ 0.209896] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)908worker2 # [ 0.212604] NET: Registered PF_UNIX/PF_LOCAL protocol family909worker2 # [ 0.212626] NET: Registered PF_XDP protocol family910lb # [ 0.210007] NET: Registered PF_UNIX/PF_LOCAL protocol family911worker2 # [ 0.212645] PCI: CLS 0 bytes, default 64912lb # [ 0.210027] NET: Registered PF_XDP protocol family913lb # [ 0.210045] PCI: CLS 0 bytes, default 64914worker2 # [ 0.212890] Trying to unpack rootfs image as initramfs...915client # [ 0.232181] kvm [1]: HYP mode not available916lb # [ 0.210249] Trying to unpack rootfs image as initramfs...917worker2 # [ 0.229188] kvm [1]: HYP mode not available918lb # [ 0.227969] kvm [1]: HYP mode not available919worker1 # [ 0.325473] Initialise system trusted keyrings920worker1 # [ 0.326231] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker1 # [ 0.327564] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker1 # [ 0.328350] 9p: Installing v9fs 9p2000 file system support923client # [ 0.327288] Initialise system trusted keyrings924client # [ 0.328021] workingset: timestamp_bits=42 max_order=19 bucket_order=0925client # [ 0.329281] squashfs: version 4.0 (2009/01/31) Phillip Lougher926client # [ 0.330066] 9p: Installing v9fs 9p2000 file system support927worker2 # [ 0.326954] Initialise system trusted keyrings928worker2 # [ 0.327743] workingset: timestamp_bits=42 max_order=19 bucket_order=0929lb # [ 0.326644] Initialise system trusted keyrings930worker2 # [ 0.329046] squashfs: version 4.0 (2009/01/31) Phillip Lougher931lb # [ 0.327405] workingset: timestamp_bits=42 max_order=19 bucket_order=0932worker1 # [ 0.357105] Key type asymmetric registered933lb # [ 0.328652] squashfs: version 4.0 (2009/01/31) Phillip Lougher934worker1 # [ 0.357133] Asymmetric key parser 'x509' registered935worker1 # [ 0.357215] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)936lb # [ 0.329421] 9p: Installing v9fs 9p2000 file system support937worker1 # [ 0.359329] io scheduler mq-deadline registered938worker1 # [ 0.359340] io scheduler kyber registered939worker1 # [ 0.369057] pl061_gpio 9030000.pl061: PL061 GPIO chip registered940worker2 # [ 0.335587] 9p: Installing v9fs 9p2000 file system support941worker1 # [ 0.370455] ledtrig-cpu: registered to indicate activity on CPUs942worker1 # [ 0.370834] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:943worker1 # [ 0.370850] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000944worker1 # [ 0.370864] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000945client # [ 0.358848] Key type asymmetric registered946client # [ 0.358875] Asymmetric key parser 'x509' registered947worker1 # [ 0.370872] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000948client # [ 0.358953] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)949worker1 # [ 0.370894] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits950client # [ 0.361134] io scheduler mq-deadline registered951client # [ 0.361146] io scheduler kyber registered952lb # [ 0.350103] Key type asymmetric registered953worker1 # [ 0.370925] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]954lb # [ 0.350130] Asymmetric key parser 'x509' registered955worker1 # [ 0.370999] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00956worker1 # [ 0.371008] pci_bus 0000:00: root bus resource [bus 00-ff]957worker1 # [ 0.371014] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]958worker1 # [ 0.371019] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]959worker2 # [ 0.355640] Key type asymmetric registered960worker1 # [ 0.371024] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]961worker2 # [ 0.355661] Asymmetric key parser 'x509' registered962worker2 # [ 0.355746] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)963worker1 # [ 0.371077] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint964worker2 # [ 0.357929] io scheduler mq-deadline registered965worker1 # [ 0.371532] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint966worker2 # [ 0.357940] io scheduler kyber registered967worker1 # [ 0.371716] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]968worker1 # [ 0.371733] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]969worker1 # [ 0.371763] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]970lb # [ 0.358254] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)971worker1 # [ 0.371779] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]972lb # [ 0.359327] io scheduler mq-deadline registered973lb # [ 0.359337] io scheduler kyber registered974client # [ 0.370822] pl061_gpio 9030000.pl061: PL061 GPIO chip registered975worker1 # [ 0.372232] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint976worker1 # [ 0.372413] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]977worker1 # [ 0.372429] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]978worker1 # [ 0.372458] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]979client # [ 0.372226] ledtrig-cpu: registered to indicate activity on CPUs980client # [ 0.372645] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:981worker1 # [ 0.391918] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint982worker1 # [ 0.392105] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]983client # [ 0.372662] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000984worker1 # [ 0.392122] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]985client # [ 0.372674] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000986worker1 # [ 0.392152] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]987client # [ 0.372683] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000988worker1 # [ 0.392621] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint989worker1 # [ 0.392805] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]990client # [ 0.372704] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits991worker1 # [ 0.392821] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]992client # [ 0.372730] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]993worker1 # [ 0.392851] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994client # [ 0.372807] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00995worker1 # [ 0.392871] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]996client # [ 0.372816] pci_bus 0000:00: root bus resource [bus 00-ff]997lb # [ 0.370314] pl061_gpio 9030000.pl061: PL061 GPIO chip registered998client # [ 0.372823] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]999client # [ 0.372828] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1000client # [ 0.372833] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1001client # [ 0.372931] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1002lb # [ 0.371752] ledtrig-cpu: registered to indicate activity on CPUs1003lb # [ 0.372147] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1004client # [ 0.373387] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1005client # [ 0.373576] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1006lb # [ 0.372164] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001007client # [ 0.373593] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1008lb # [ 0.372177] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001009client # [ 0.373624] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1010worker2 # [ 0.371709] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1011client # [ 0.373641] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1012worker2 # [ 0.372395] ledtrig-cpu: registered to indicate activity on CPUs1013lb # [ 0.372186] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001014worker2 # [ 0.372800] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1015client # [ 0.374119] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1016lb # [ 0.372206] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1017client # [ 0.374306] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1018worker2 # [ 0.372818] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001019worker1 # [ 0.405425] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1020lb # [ 0.372232] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1021client # [ 0.374322] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1022worker1 # [ 0.405633] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1023lb # [ 0.372311] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001024worker2 # [ 0.372830] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001025client # [ 0.374352] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1026worker1 # [ 0.405664] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1027lb # [ 0.372320] pci_bus 0000:00: root bus resource [bus 00-ff]1028worker2 # [ 0.372839] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001029lb # [ 0.372326] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1030worker1 # [ 0.406129] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1031client # [ 0.393884] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1032worker2 # [ 0.372861] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1033lb # [ 0.372332] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1034worker1 # [ 0.406318] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1035client # [ 0.394075] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1036worker1 # [ 0.406348] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037worker2 # [ 0.372887] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1038lb # [ 0.372337] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1039client # [ 0.394096] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1040worker2 # [ 0.372973] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001041lb # [ 0.372397] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1042worker1 # [ 0.406742] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1043client # [ 0.394126] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044worker2 # [ 0.372984] pci_bus 0000:00: root bus resource [bus 00-ff]1045worker1 # [ 0.406924] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1046lb # [ 0.372833] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1047client # [ 0.394597] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1048worker2 # [ 0.372990] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1049lb # [ 0.373020] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1050worker1 # [ 0.407215] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1051worker2 # [ 0.372995] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1052lb # [ 0.373036] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1053worker1 # [ 0.407416] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1054worker2 # [ 0.373000] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1055worker1 # [ 0.407446] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1056lb # [ 0.373066] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057lb # [ 0.373083] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1058worker2 # [ 0.373083] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1059worker1 # [ 0.407905] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1060worker2 # [ 0.373535] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1061worker1 # [ 0.408094] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1062lb # [ 0.373533] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1063lb # [ 0.373716] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1064worker2 # [ 0.373724] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1065worker1 # [ 0.408124] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1066lb # [ 0.373732] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1067worker2 # [ 0.373740] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1068worker1 # [ 0.408576] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1069worker2 # [ 0.373771] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070lb # [ 0.373761] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1071worker1 # [ 0.408764] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1072worker2 # [ 0.373787] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1073worker1 # [ 0.408794] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074worker2 # [ 0.374260] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1075worker2 # [ 0.374447] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1076lb # [ 0.392793] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1077lb # [ 0.392980] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1078worker2 # [ 0.374463] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1079lb # [ 0.392996] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1080worker2 # [ 0.374492] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081worker1 # [ 0.423391] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1082lb # [ 0.393027] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1083worker1 # [ 0.423678] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1084worker2 # [ 0.374947] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1085worker1 # [ 0.423696] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1086worker2 # [ 0.375134] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1087lb # [ 0.393488] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1088lb # [ 0.393674] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1089worker2 # [ 0.375149] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1090worker1 # [ 0.423726] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1091lb # [ 0.393690] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1092client # [ 0.402851] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1093worker2 # [ 0.375180] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094worker1 # [ 0.424188] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1095client # [ 0.402883] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1096lb # [ 0.393719] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1097worker1 # [ 0.424371] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1098worker2 # [ 0.375663] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1099lb # [ 0.393737] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1100client # [ 0.402914] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101worker2 # [ 0.375850] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1102worker1 # [ 0.424388] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1103client # [ 0.402933] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1104worker2 # [ 0.375866] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1105worker1 # [ 0.424418] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1106client # [ 0.403432] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1107worker2 # [ 0.375896] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108client # [ 0.403626] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1109worker2 # [ 0.375915] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1110client # [ 0.403656] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111worker2 # [ 0.376389] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1112worker2 # [ 0.376588] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1113client # [ 0.404119] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1114client # [ 0.404309] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1115worker2 # [ 0.376618] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1116client # [ 0.404340] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1117worker2 # [ 0.377124] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1118worker2 # [ 0.377318] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1119client # [ 0.404731] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1120worker2 # [ 0.377348] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1121client # [ 0.404915] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1122worker2 # [ 0.377744] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1123client # [ 0.405204] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1124worker2 # [ 0.377930] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1125client # [ 0.405398] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1126client # [ 0.405428] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127worker2 # [ 0.378178] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1128worker2 # [ 0.378369] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1129client # [ 0.405896] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1130worker1 # [ 0.437111] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1131worker2 # [ 0.378399] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1132client # [ 0.406086] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1133worker1 # [ 0.437135] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1134client # [ 0.406129] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135worker2 # [ 0.378856] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1136worker2 # [ 0.379045] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1137worker1 # [ 0.437141] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1138client # [ 0.406589] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1139lb # [ 0.406261] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1140client # [ 0.406798] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1141worker2 # [ 0.379075] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1142worker1 # [ 0.437194] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1143lb # [ 0.406487] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1144client # [ 0.406829] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145worker1 # [ 0.437242] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1146lb # [ 0.406520] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147client # [ 0.407294] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1148worker1 # [ 0.437292] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1149lb # [ 0.407037] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1150client # [ 0.407594] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1151lb # [ 0.407229] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1152client # [ 0.407614] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1153worker1 # [ 0.437341] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1154lb # [ 0.407259] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1155client # [ 0.407644] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1156worker1 # [ 0.437389] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1157lb # [ 0.407652] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1158client # [ 0.408132] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1159worker1 # [ 0.437437] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1160client # [ 0.408317] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1161lb # [ 0.407837] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1162worker1 # [ 0.437490] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1163client # [ 0.408334] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1164lb # [ 0.408093] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1165worker1 # [ 0.437539] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1166client # [ 0.408364] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167lb # [ 0.408283] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1168client # [ 0.408955] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1169worker1 # [ 0.437587] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1170lb # [ 0.408313] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171client # [ 0.408967] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1172worker1 # [ 0.437704] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1173lb # [ 0.408765] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1174client # [ 0.408972] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1175worker1 # [ 0.437753] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1176lb # [ 0.408952] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1177worker1 # [ 0.437775] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1178client # [ 0.409018] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1179lb # [ 0.408982] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180worker1 # [ 0.437798] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1181client # [ 0.409066] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1182lb # [ 0.409436] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1183worker1 # [ 0.437820] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1184lb # [ 0.409624] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1185client # [ 0.409114] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1186worker1 # [ 0.437842] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1187lb # [ 0.409654] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1188client # [ 0.409161] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1189worker2 # [ 0.423607] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1190worker1 # [ 0.437864] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1191lb # [ 0.410119] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1192worker2 # [ 0.423828] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1193client # [ 0.409208] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1194worker1 # [ 0.437887] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1195lb # [ 0.410441] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1196worker2 # [ 0.423860] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1197client # [ 0.409255] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1198lb # [ 0.410459] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1199worker1 # [ 0.437914] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1200worker2 # [ 0.424405] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1201worker1 # [ 0.437937] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1202lb # [ 0.410489] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1203client # [ 0.409302] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1204worker2 # [ 0.424706] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1205worker1 # [ 0.437959] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1206lb # [ 0.410965] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1207client # [ 0.409349] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1208worker2 # [ 0.424724] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1209worker1 # [ 0.437982] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1210lb # [ 0.411154] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1211client # [ 0.409396] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1212worker2 # [ 0.424754] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1213lb # [ 0.411170] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1214worker1 # [ 0.438005] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1215client # [ 0.409466] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1216worker1 # [ 0.438026] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1217lb # [ 0.411209] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1218worker2 # [ 0.425228] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1219worker2 # [ 0.425416] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1220client # [ 0.409513] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1221lb # [ 0.411787] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1222worker1 # [ 0.438048] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1223worker2 # [ 0.425433] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1224client # [ 0.409536] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1225worker1 # [ 0.438074] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1226lb # [ 0.411798] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1227worker1 # [ 0.438096] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1228worker2 # [ 0.425463] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1229client # [ 0.409558] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1230lb # [ 0.411804] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1231worker1 # [ 0.438118] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1232worker2 # [ 0.426049] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1233client # [ 0.409580] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1234lb # [ 0.411850] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1235worker1 # [ 0.438142] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1236worker2 # [ 0.426061] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1237client # [ 0.409601] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1238worker1 # [ 0.438171] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1239lb # [ 0.411898] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1240client # [ 0.409624] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1241worker2 # [ 0.426067] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1242worker1 # [ 0.438181] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1243client # [ 0.409646] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1244lb # [ 0.411946] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1245worker1 # [ 0.438185] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1246worker2 # [ 0.426114] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1247client # [ 0.409671] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1248worker1 # [ 0.439018] pci 0000:00:07.0: enabling device (0000 -> 0002)1249lb # [ 0.411998] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1250worker2 # [ 0.426163] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1251client # [ 0.409693] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1252lb # [ 0.412044] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1253worker2 # [ 0.426210] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1254client # [ 0.409716] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1255lb # [ 0.412091] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1256worker2 # [ 0.426259] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1257client # [ 0.409739] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1258lb # [ 0.412138] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1259client # [ 0.409762] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1260worker2 # [ 0.426307] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1261client # [ 0.409784] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1262lb # [ 0.412184] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1263worker2 # [ 0.426356] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1264client # [ 0.409806] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1265lb # [ 0.412231] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1266worker2 # [ 0.426403] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1267client # [ 0.409827] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1268lb # [ 0.412307] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1269client # [ 0.409849] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1270worker2 # [ 0.426451] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1271lb # [ 0.412353] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1272client # [ 0.409871] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1273worker2 # [ 0.426497] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1274lb # [ 0.412375] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1275client # [ 0.409892] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1276worker2 # [ 0.426567] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1277lb # [ 0.412396] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1278client # [ 0.409918] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1279worker2 # [ 0.426616] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1280client # [ 0.409928] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1281lb # [ 0.412417] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1282worker2 # [ 0.426638] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1283client # [ 0.409932] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1284lb # [ 0.412438] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1285worker2 # [ 0.426660] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1286lb # [ 0.412460] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1287worker2 # [ 0.426681] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1288lb # [ 0.412481] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1289worker2 # [ 0.426703] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1290lb # [ 0.412506] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1291worker2 # [ 0.426725] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1292lb # [ 0.412528] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1293worker2 # [ 0.426747] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1294lb # [ 0.412550] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1295worker2 # [ 0.426773] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1296lb # [ 0.412572] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1297worker2 # [ 0.426796] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1298lb # [ 0.412595] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1299worker2 # [ 0.426818] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1300lb # [ 0.412616] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1301lb # [ 0.412637] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1302worker2 # [ 0.426840] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1303client # [ 0.466823] pci 0000:00:07.0: enabling device (0000 -> 0002)1304lb # [ 0.412658] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1305worker2 # [ 0.426863] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1306lb # [ 0.412679] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1307worker2 # [ 0.426884] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1308lb # [ 0.412700] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1309worker2 # [ 0.426906] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1310lb # [ 0.412721] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1311worker2 # [ 0.426928] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1312worker1 # [ 0.488105] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1313lb # [ 0.412748] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1314worker2 # [ 0.426949] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1315lb # [ 0.412758] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1316worker2 # [ 0.426970] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1317lb # [ 0.412763] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1318worker2 # [ 0.426991] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1319lb # [ 0.413591] pci 0000:00:07.0: enabling device (0000 -> 0002)1320worker2 # [ 0.427018] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1321worker2 # [ 0.427028] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1322worker2 # [ 0.427032] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1323worker2 # [ 0.427882] pci 0000:00:07.0: enabling device (0000 -> 0002)1324worker1 # [ 0.494316] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1325worker1 # [ 0.496298] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1326worker1 # [ 0.506384] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1327worker1 # [ 0.508382] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1328client # [ 0.493157] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1329lb # [ 0.490042] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1330client # [ 0.499850] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1331worker1 # [ 0.519977] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1332client # [ 0.501912] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1333worker2 # [ 0.494051] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1334worker1 # [ 0.522598] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1335worker1 # [ 0.524522] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1336worker2 # [ 0.497070] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1337worker2 # [ 0.499119] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1338client # [ 0.512140] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1339lb # [ 0.499510] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1340client # [ 0.514308] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1341lb # [ 0.501877] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1342worker1 # [ 0.534360] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1343worker1 # [ 0.536319] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1344worker2 # [ 0.509013] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1345worker2 # [ 0.511271] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1346client # [ 0.525628] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1347lb # [ 0.511462] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1348lb # [ 0.514098] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1349client # [ 0.528194] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1350client # [ 0.529928] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1351worker1 # [ 0.548081] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1352worker2 # [ 0.522662] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1353worker2 # [ 0.525227] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1354lb # [ 0.523816] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1355worker2 # [ 0.526971] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1356worker1 # [ 0.558103] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1357lb # [ 0.527254] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1358lb # [ 0.529221] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1359client # [ 0.540009] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1360client # [ 0.542050] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1361client # [ 0.546152] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1362worker1 # [ 0.561079] msm_serial: driver initialized1363worker1 # [ 0.561209] SuperH (H)SCI(F) driver initialized1364worker1 # [ 0.561263] STM32 USART driver initialized1365worker2 # [ 0.536988] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1366worker2 # [ 0.539181] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1367worker2 # [ 0.543009] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1368lb # [ 0.539278] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1369lb # [ 0.541430] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1370lb # [ 0.545374] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1371client # [ 0.563964] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1372client # [ 0.566708] msm_serial: driver initialized1373client # [ 0.566870] SuperH (H)SCI(F) driver initialized1374worker2 # [ 0.560778] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1375client # [ 0.566923] STM32 USART driver initialized1376lb # [ 0.563389] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1377worker1 # [ 0.595774] loop: module loaded1378worker1 # [ 0.595960] virtio_blk virtio2: 1/0/0 default/read/poll queues1379worker2 # [ 0.568884] msm_serial: driver initialized1380worker2 # [ 0.569055] SuperH (H)SCI(F) driver initialized1381worker1 # [ 0.596798] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1382worker2 # [ 0.569110] STM32 USART driver initialized1383lb # [ 0.571563] msm_serial: driver initialized1384lb # [ 0.571706] SuperH (H)SCI(F) driver initialized1385lb # [ 0.571762] STM32 USART driver initialized1386worker1 # [ 0.605023] megasas: 07.734.00.00-rc11387worker1 # [ 0.605792] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1388worker1 # [ 0.608111] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001389worker1 # [ 0.608138] Intel/Sharp Extended Query Table at 0x00311390worker1 # [ 0.609755] Using buffer write method1391worker1 # [ 0.609827] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1392worker1 # [ 0.611572] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001393client # [ 0.601439] loop: module loaded1394worker1 # [ 0.611594] Intel/Sharp Extended Query Table at 0x00311395client # [ 0.601635] virtio_blk virtio2: 1/0/0 default/read/poll queues1396client # [ 0.603820] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1397worker2 # [ 0.598385] loop: module loaded1398worker2 # [ 0.598578] virtio_blk virtio2: 1/0/0 default/read/poll queues1399client # [ 0.607277] megasas: 07.734.00.00-rc11400worker2 # [ 0.599378] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1401client # [ 0.607964] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1402worker1 # [ 0.630282] Using buffer write method1403worker1 # [ 0.630317] Concatenating MTD devices:1404worker1 # [ 0.630321] (0): "0.flash"1405lb # [ 0.601202] loop: module loaded1406worker1 # [ 0.630324] (1): "0.flash"1407worker1 # [ 0.630335] into device "0.flash"1408lb # [ 0.601409] virtio_blk virtio2: 1/0/0 default/read/poll queues1409lb # [ 0.603682] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1410client # [ 0.619283] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001411client # [ 0.619329] Intel/Sharp Extended Query Table at 0x00311412client # [ 0.621004] Using buffer write method1413client # [ 0.621070] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1414worker2 # [ 0.612214] megasas: 07.734.00.00-rc11415worker2 # [ 0.613041] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1416worker2 # [ 0.614976] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417worker2 # [ 0.615007] Intel/Sharp Extended Query Table at 0x00311418worker2 # [ 0.616812] Using buffer write method1419lb # [ 0.614850] megasas: 07.734.00.00-rc11420worker2 # [ 0.616894] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1421lb # [ 0.615596] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1422worker2 # [ 0.618695] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001423lb # [ 0.617760] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001424worker2 # [ 0.618726] Intel/Sharp Extended Query Table at 0x00311425lb # [ 0.617785] Intel/Sharp Extended Query Table at 0x00311426lb # [ 0.619371] Using buffer write method1427lb # [ 0.619436] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1428client # [ 0.636532] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001429client # [ 0.636564] Intel/Sharp Extended Query Table at 0x00311430client # [ 0.638168] Using buffer write method1431lb # [ 0.621056] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001432client # [ 0.638194] Concatenating MTD devices:1433client # [ 0.638198] (0): "0.flash"1434lb # [ 0.621078] Intel/Sharp Extended Query Table at 0x00311435client # [ 0.638202] (1): "0.flash"1436client # [ 0.638210] into device "0.flash"1437worker2 # [ 0.639225] Using buffer write method1438worker2 # [ 0.639268] Concatenating MTD devices:1439worker2 # [ 0.639272] (0): "0.flash"1440worker2 # [ 0.639277] (1): "0.flash"1441worker2 # [ 0.639285] into device "0.flash"1442lb # [ 0.638801] Using buffer write method1443lb # [ 0.638835] Concatenating MTD devices:1444lb # [ 0.638840] (0): "0.flash"1445lb # [ 0.638844] (1): "0.flash"1446lb # [ 0.638852] into device "0.flash"1447worker1 # [ 0.883975] Freeing initrd memory: 26976K1448worker1 # [ 0.889940] tun: Universal TUN/TAP device driver, 1.61449worker1 # [ 0.893771] thunder_xcv, ver 1.01450worker1 # [ 0.893812] thunder_bgx, ver 1.01451worker1 # [ 0.893834] nicpf, ver 1.01452worker1 # [ 0.894368] e1000: Intel(R) PRO/1000 Network Driver1453worker1 # [ 0.894375] e1000: Copyright (c) 1999-2006 Intel Corporation.1454worker1 # [ 0.894399] e1000e: Intel(R) PRO/1000 Network Driver1455worker1 # [ 0.894407] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1456worker1 # [ 0.894436] igb: Intel(R) Gigabit Ethernet Network Driver1457worker1 # [ 0.894442] igb: Copyright (c) 2007-2014 Intel Corporation.1458worker1 # [ 0.894463] igbvf: Intel(R) Gigabit Virtual Function Network Driver1459worker1 # [ 0.894469] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1460worker1 # [ 0.894599] sky2: driver version 1.301461worker1 # [ 0.896203] usbcore: registered new interface driver usb-storage1462worker1 # [ 0.896251] usbcore: registered new interface driver usbserial_generic1463worker1 # [ 0.896265] usbserial: USB Serial support registered for generic1464worker1 # [ 0.896848] hv_vmbus: registering driver hyperv_keyboard1465worker1 # [ 0.897806] ehci-pci 0000:00:07.0: EHCI Host Controller1466worker1 # [ 0.897832] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11467worker1 # [ 0.898065] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001468worker1 # [ 0.910512] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001469client # [ 0.894309] Freeing initrd memory: 26972K1470worker1 # [ 0.910863] hub 1-0:1.0: USB hub found1471worker1 # [ 0.910886] hub 1-0:1.0: 6 ports detected1472worker1 # [ 0.913880] rtc-pl031 9010000.pl031: registered as rtc01473worker1 # [ 0.913910] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T15:45:13 UTC (1790091913)1474worker1 # [ 0.914211] i2c_dev: i2c /dev entries driver1475worker2 # [ 0.888616] Freeing initrd memory: 26944K1476client # [ 0.900313] tun: Universal TUN/TAP device driver, 1.61477worker1 # [ 0.919637] sdhci: Secure Digital Host Controller Interface driver1478worker1 # [ 0.919649] sdhci: Copyright(c) Pierre Ossman1479worker2 # [ 0.894757] tun: Universal TUN/TAP device driver, 1.61480worker1 # [ 0.919915] Synopsys Designware Multimedia Card Interface Driver1481worker1 # [ 0.920286] sdhci-pltfm: SDHCI platform and OF driver helper1482client # [ 0.904141] thunder_xcv, ver 1.01483worker1 # [ 0.924593] hid: raw HID events driver (C) Jiri Kosina1484lb # [ 0.894564] Freeing initrd memory: 26948K1485client # [ 0.904186] thunder_bgx, ver 1.01486worker1 # [ 0.924858] usbcore: registered new interface driver usbhid1487client # [ 0.904208] nicpf, ver 1.01488worker1 # [ 0.924864] usbhid: USB HID core driver1489client # [ 0.904754] e1000: Intel(R) PRO/1000 Network Driver1490worker2 # [ 0.898755] thunder_xcv, ver 1.01491worker2 # [ 0.898801] thunder_bgx, ver 1.01492client # [ 0.904762] e1000: Copyright (c) 1999-2006 Intel Corporation.1493worker2 # [ 0.898824] nicpf, ver 1.01494client # [ 0.904785] e1000e: Intel(R) PRO/1000 Network Driver1495worker2 # [ 0.899398] e1000: Intel(R) PRO/1000 Network Driver1496client # [ 0.904793] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1497worker2 # [ 0.899406] e1000: Copyright (c) 1999-2006 Intel Corporation.1498client # [ 0.904823] igb: Intel(R) Gigabit Ethernet Network Driver1499worker2 # [ 0.899430] e1000e: Intel(R) PRO/1000 Network Driver1500worker1 # [ 0.927782] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1501client # [ 0.904829] igb: Copyright (c) 2007-2014 Intel Corporation.1502lb # [ 0.900493] tun: Universal TUN/TAP device driver, 1.61503worker2 # [ 0.899438] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1504client # [ 0.904855] igbvf: Intel(R) Gigabit Virtual Function Network Driver1505worker2 # [ 0.899467] igb: Intel(R) Gigabit Ethernet Network Driver1506worker1 # [ 0.930334] drop_monitor: Initializing network drop monitor service1507client # [ 0.904861] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1508worker2 # [ 0.899473] igb: Copyright (c) 2007-2014 Intel Corporation.1509client # [ 0.904991] sky2: driver version 1.301510worker1 # [ 0.930523] NET: Registered PF_INET6 protocol family1511worker2 # [ 0.899496] igbvf: Intel(R) Gigabit Virtual Function Network Driver1512client # [ 0.906569] usbcore: registered new interface driver usb-storage1513worker1 # [ 0.932400] Segment Routing with IPv61514worker2 # [ 0.899502] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1515client # [ 0.907446] ehci-pci 0000:00:07.0: EHCI Host Controller1516worker1 # [ 0.932418] In-situ OAM (IOAM) with IPv61517worker1 # [ 0.932447] NET: Registered PF_PACKET protocol family1518client # [ 0.907471] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11519client # [ 0.907739] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001520worker2 # [ 0.906223] sky2: driver version 1.301521lb # [ 0.904151] thunder_xcv, ver 1.01522worker2 # [ 0.908167] ehci-pci 0000:00:07.0: EHCI Host Controller1523lb # [ 0.904194] thunder_bgx, ver 1.01524lb # [ 0.904216] nicpf, ver 1.01525worker1 # [ 0.934264] 9pnet: Installing 9P2000 support1526worker2 # [ 0.908196] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11527client # [ 0.917545] usbcore: registered new interface driver usbserial_generic1528lb # [ 0.904754] e1000: Intel(R) PRO/1000 Network Driver1529worker1 # [ 0.934312] Key type dns_resolver registered1530worker2 # [ 0.908454] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001531client # [ 0.917565] usbserial: USB Serial support registered for generic1532lb # [ 0.904763] e1000: Copyright (c) 1999-2006 Intel Corporation.1533client # [ 0.918286] hv_vmbus: registering driver hyperv_keyboard1534lb # [ 0.904786] e1000e: Intel(R) PRO/1000 Network Driver1535client # [ 0.918716] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001536client # [ 0.919030] hub 1-0:1.0: USB hub found1537lb # [ 0.904795] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1538client # [ 0.919087] hub 1-0:1.0: 6 ports detected1539lb # [ 0.904823] igb: Intel(R) Gigabit Ethernet Network Driver1540worker2 # [ 0.911563] usbcore: registered new interface driver usb-storage1541lb # [ 0.904829] igb: Copyright (c) 2007-2014 Intel Corporation.1542worker2 # [ 0.911616] usbcore: registered new interface driver usbserial_generic1543lb # [ 0.904868] igbvf: Intel(R) Gigabit Virtual Function Network Driver1544worker2 # [ 0.911630] usbserial: USB Serial support registered for generic1545client # [ 0.923613] rtc-pl031 9010000.pl031: registered as rtc01546lb # [ 0.904874] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1547worker2 # [ 0.912226] hv_vmbus: registering driver hyperv_keyboard1548lb # [ 0.905008] sky2: driver version 1.301549client # [ 0.923646] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T15:45:13 UTC (1790091913)1550lb # [ 0.906917] ehci-pci 0000:00:07.0: EHCI Host Controller1551client # [ 0.923957] i2c_dev: i2c /dev entries driver1552lb # [ 0.906945] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11553worker2 # [ 0.916041] rtc-pl031 9010000.pl031: registered as rtc01554lb # [ 0.907114] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001555worker1 # [ 0.941289] registered taskstats version 11556worker2 # [ 0.916070] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T15:45:13 UTC (1790091913)1557worker1 # [ 0.941425] Loading compiled-in X.509 certificates1558worker2 # [ 0.916449] i2c_dev: i2c /dev entries driver1559worker2 # [ 0.919564] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001560worker2 # [ 0.919851] hub 1-0:1.0: USB hub found1561lb # [ 0.916921] usbcore: registered new interface driver usb-storage1562client # [ 0.929238] sdhci: Secure Digital Host Controller Interface driver1563worker2 # [ 0.919872] hub 1-0:1.0: 6 ports detected1564client # [ 0.929248] sdhci: Copyright(c) Pierre Ossman1565lb # [ 0.917024] usbcore: registered new interface driver usbserial_generic1566client # [ 0.929505] Synopsys Designware Multimedia Card Interface Driver1567lb # [ 0.917038] usbserial: USB Serial support registered for generic1568client # [ 0.929875] sdhci-pltfm: SDHCI platform and OF driver helper1569lb # [ 0.917631] hv_vmbus: registering driver hyperv_keyboard1570worker1 # [ 0.949981] Demotion targets for Node 0: null1571lb # [ 0.918214] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001572worker1 # [ 0.950082] Key type .fscrypt registered1573lb # [ 0.918512] hub 1-0:1.0: USB hub found1574worker1 # [ 0.950089] Key type fscrypt-provisioning registered1575lb # [ 0.918532] hub 1-0:1.0: 6 ports detected1576client # [ 0.934036] hid: raw HID events driver (C) Jiri Kosina1577worker1 # [ 0.950187] ima: No TPM chip found, activating TPM-bypass!1578client # [ 0.934275] usbcore: registered new interface driver usbhid1579worker1 # [ 0.950205] ima: Allocated hash algorithm: sha11580client # [ 0.934282] usbhid: USB HID core driver1581worker1 # [ 0.950228] ima: No architecture policies found1582lb # [ 0.923658] rtc-pl031 9010000.pl031: registered as rtc01583worker1 # [ 0.954190] input: gpio-keys as /devices/platform/gpio-keys/input/input01584worker2 # [ 0.924234] sdhci: Secure Digital Host Controller Interface driver1585lb # [ 0.923687] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T15:45:13 UTC (1790091913)1586worker2 # [ 0.924247] sdhci: Copyright(c) Pierre Ossman1587lb # [ 0.924003] i2c_dev: i2c /dev entries driver1588worker2 # [ 0.924524] Synopsys Designware Multimedia Card Interface Driver1589client # [ 0.937106] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1590worker2 # [ 0.924898] sdhci-pltfm: SDHCI platform and OF driver helper1591client # [ 0.938608] drop_monitor: Initializing network drop monitor service1592worker2 # [ 0.926421] hid: raw HID events driver (C) Jiri Kosina1593client # [ 0.940588] NET: Registered PF_INET6 protocol family1594worker2 # [ 0.926689] usbcore: registered new interface driver usbhid1595worker2 # [ 0.926695] usbhid: USB HID core driver1596client # [ 0.941727] Segment Routing with IPv61597lb # [ 0.929353] sdhci: Secure Digital Host Controller Interface driver1598client # [ 0.941746] In-situ OAM (IOAM) with IPv61599lb # [ 0.929368] sdhci: Copyright(c) Pierre Ossman1600client # [ 0.941810] NET: Registered PF_PACKET protocol family1601lb # [ 0.929630] Synopsys Designware Multimedia Card Interface Driver1602lb # [ 0.930005] sdhci-pltfm: SDHCI platform and OF driver helper1603worker2 # [ 0.932455] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1604worker2 # [ 0.934013] drop_monitor: Initializing network drop monitor service1605worker2 # [ 0.934158] NET: Registered PF_INET6 protocol family1606client # [ 0.943631] 9pnet: Installing 9P2000 support1607worker2 # [ 0.937215] Segment Routing with IPv61608client # [ 0.943675] Key type dns_resolver registered1609lb # [ 0.934271] hid: raw HID events driver (C) Jiri Kosina1610worker2 # [ 0.937235] In-situ OAM (IOAM) with IPv61611lb # [ 0.934528] usbcore: registered new interface driver usbhid1612worker2 # [ 0.937268] NET: Registered PF_PACKET protocol family1613lb # [ 0.934535] usbhid: USB HID core driver1614worker2 # [ 0.938935] 9pnet: Installing 9P2000 support1615worker2 # [ 0.938986] Key type dns_resolver registered1616lb # [ 0.937409] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1617client # [ 0.950278] registered taskstats version 11618client # [ 0.950439] Loading compiled-in X.509 certificates1619lb # [ 0.940081] drop_monitor: Initializing network drop monitor service1620lb # [ 0.940227] NET: Registered PF_INET6 protocol family1621lb # [ 0.942126] Segment Routing with IPv61622lb # [ 0.942157] In-situ OAM (IOAM) with IPv61623lb # [ 0.943336] NET: Registered PF_PACKET protocol family1624worker1 # [ 0.972069] clk: Disabling unused clocks1625worker1 # [ 0.972098] PM: genpd: Disabling unused power domains1626worker2 # [ 0.945670] registered taskstats version 11627worker2 # [ 0.945830] Loading compiled-in X.509 certificates1628lb # [ 0.944059] 9pnet: Installing 9P2000 support1629lb # [ 0.944104] Key type dns_resolver registered1630worker1 # [ 0.976242] Freeing unused kernel memory: 4736K1631client # [ 0.959116] Demotion targets for Node 0: null1632worker1 # [ 0.976441] Run /init as init process1633client # [ 0.959224] Key type .fscrypt registered1634client # [ 0.959230] Key type fscrypt-provisioning registered1635client # [ 0.959319] ima: No TPM chip found, activating TPM-bypass!1636client # [ 0.959338] ima: Allocated hash algorithm: sha11637client # [ 0.959360] ima: No architecture policies found1638client # [ 0.963560] input: gpio-keys as /devices/platform/gpio-keys/input/input01639worker2 # [ 0.954491] Demotion targets for Node 0: null1640worker2 # [ 0.954610] Key type .fscrypt registered1641worker2 # [ 0.954617] Key type fscrypt-provisioning registered1642lb # [ 0.950602] registered taskstats version 11643worker2 # [ 0.954721] ima: No TPM chip found, activating TPM-bypass!1644lb # [ 0.950741] Loading compiled-in X.509 certificates1645worker2 # [ 0.954740] ima: Allocated hash algorithm: sha11646worker2 # [ 0.954763] ima: No architecture policies found1647worker2 # [ 0.958862] input: gpio-keys as /devices/platform/gpio-keys/input/input01648lb # [ 0.959415] Demotion targets for Node 0: null1649lb # [ 0.959527] Key type .fscrypt registered1650lb # [ 0.959534] Key type fscrypt-provisioning registered1651lb # [ 0.959626] ima: No TPM chip found, activating TPM-bypass!1652lb # [ 0.959646] ima: Allocated hash algorithm: sha11653worker1 # [ 0.991175] systemd[1]: Successfully made /usr/ read-only.1654lb # [ 0.959669] ima: No architecture policies found1655lb # [ 0.963721] input: gpio-keys as /devices/platform/gpio-keys/input/input01656client # [ 0.980996] clk: Disabling unused clocks1657client # [ 0.981027] PM: genpd: Disabling unused power domains1658client # [ 0.985152] Freeing unused kernel memory: 4736K1659client # [ 0.985347] Run /init as init process1660worker2 # [ 0.976648] clk: Disabling unused clocks1661worker2 # [ 0.976680] PM: genpd: Disabling unused power domains1662worker2 # [ 0.980891] Freeing unused kernel memory: 4736K1663worker2 # [ 0.981074] Run /init as init process1664lb # [ 0.982307] clk: Disabling unused clocks1665lb # [ 0.982339] PM: genpd: Disabling unused power domains1666lb # [ 0.986593] Freeing unused kernel memory: 4736K1667lb # [ 0.986803] Run /init as init process1668client # [ 1.000901] systemd[1]: Successfully made /usr/ read-only.1669worker2 # [ 0.997781] systemd[1]: Successfully made /usr/ read-only.1670lb # [ 1.004383] systemd[1]: Successfully made /usr/ read-only.1671worker1 # [ 1.156981] usb 1-1: new high-speed USB device number 2 using ehci-pci1672client # [ 1.166717] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker2 # [ 1.167588] usb 1-1: new high-speed USB device number 2 using ehci-pci1674lb # [ 1.166235] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker1 # [ 1.309239] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker1 # [ 1.326069] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1677worker1 # [ 1.338343] systemd[1]: Detected virtualization qemu.1678client # [ 1.319229] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679worker1 # [ 1.340343] systemd[1]: Detected architecture arm64.1680worker1 # [ 1.342365] systemd[1]: Running in initrd.1681worker1 # [ 1.345048] systemd[1]: Initializing machine ID from random generator.1682worker1 # [ 1.347935] systemd[1]: Hostname set to <worker1>.1683worker2 # [ 1.320131] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684lb # [ 1.318560] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685client # [ 1.335727] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1686client # [ 1.347886] systemd[1]: Detected virtualization qemu.1687client # [ 1.349869] systemd[1]: Detected architecture arm64.1688client # [ 1.351849] systemd[1]: Running in initrd.1689worker2 # [ 1.332982] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1690worker2 # [ 1.345135] systemd[1]: Detected virtualization qemu.1691client # [ 1.354504] systemd[1]: Initializing machine ID from random generator.1692worker2 # [ 1.347233] systemd[1]: Detected architecture arm64.1693client # [ 1.357519] systemd[1]: Hostname set to <client>.1694worker2 # [ 1.349298] systemd[1]: Running in initrd.1695lb # [ 1.339131] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1696worker2 # [ 1.351897] systemd[1]: Initializing machine ID from random generator.1697lb # [ 1.351427] systemd[1]: Detected virtualization qemu.1698worker2 # [ 1.354708] systemd[1]: Hostname set to <worker2>.1699lb # [ 1.353429] systemd[1]: Detected architecture arm64.1700lb # [ 1.355441] systemd[1]: Running in initrd.1701lb # [ 1.358034] systemd[1]: Initializing machine ID from random generator.1702lb # [ 1.361019] systemd[1]: Hostname set to <lb>.1703worker1 # [ 1.393268] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704client # [ 1.406930] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705worker2 # [ 1.403815] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706lb # [ 1.410481] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker1 # [ 1.516950] usb 1-2: new high-speed USB device number 3 using ehci-pci1708client # [ 1.530732] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.523575] usb 1-2: new high-speed USB device number 3 using ehci-pci1710lb # [ 1.534229] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker1 # [ 1.661712] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712worker1 # [ 1.673914] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21713worker1 # [ 1.679545] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01714worker2 # [ 1.660577] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715client # [ 1.671950] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716lb # [ 1.670712] systemd[1]: bpf-restrict-fs: LSM BPF program attached1717client # [ 1.691472] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718worker2 # [ 1.682187] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719worker2 # [ 1.687935] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720client # [ 1.698963] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01721lb # [ 1.690705] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21722lb # [ 1.696870] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723worker1 # [ 1.770599] systemd[1]: Queued start job for default target Initrd Default Target.1724worker1 # [ 1.775072] systemd[1]: Created slice Slice /system/modprobe.1725worker1 # [ 1.776142] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1726worker1 # [ 1.777347] systemd[1]: Expecting device /dev/disk/by-label/nixos...1727worker1 # [ 1.778282] systemd[1]: Reached target Path Units.1728worker1 # [ 1.778993] systemd[1]: Reached target Slice Units.1729worker1 # [ 1.779750] systemd[1]: Reached target Swaps.1730worker1 # [ 1.780449] systemd[1]: Reached target Timer Units.1731worker1 # [ 1.781535] systemd[1]: Listening on D-Bus System Message Bus Socket.1732worker1 # [ 1.782617] systemd[1]: Listening on Journal Socket (/dev/log).1733worker1 # [ 1.783612] systemd[1]: Listening on Journal Sockets.1734worker1 # [ 1.784479] systemd[1]: Listening on udev Control Socket.1735worker1 # [ 1.785469] systemd[1]: Listening on udev Kernel Socket.1736worker1 # [ 1.785493] systemd[1]: Reached target Socket Units.1737worker1 # [ 1.788445] systemd[1]: Starting Create List of Static Device Nodes...1738worker1 # [ 1.789542] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1739worker1 # [ 1.799817] systemd[1]: Mounting Kernel Configuration File System...1740worker2 # [ 1.774747] systemd[1]: Queued start job for default target Initrd Default Target.1741client # [ 1.786698] systemd[1]: Queued start job for default target Initrd Default Target.1742lb # [ 1.783253] systemd[1]: Queued start job for default target Initrd Default Target.1743worker2 # [ 1.785889] systemd[1]: Created slice Slice /system/modprobe.1744worker2 # [ 1.787094] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1745worker2 # [ 1.788501] systemd[1]: Expecting device /dev/disk/by-label/nixos...1746client # [ 1.798567] systemd[1]: Created slice Slice /system/modprobe.1747worker2 # [ 1.789672] systemd[1]: Reached target Path Units.1748worker2 # [ 1.790495] systemd[1]: Reached target Slice Units.1749worker2 # [ 1.791327] systemd[1]: Reached target Swaps.1750worker2 # [ 1.792144] systemd[1]: Reached target Timer Units.1751client # [ 1.799912] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1752worker2 # [ 1.793180] systemd[1]: Listening on D-Bus System Message Bus Socket.1753client # [ 1.799961] systemd[1]: Expecting device /dev/disk/by-label/nixos...1754worker1 # [ 1.821130] systemd[1]: Starting Journal Service...1755worker2 # [ 1.794426] systemd[1]: Listening on Journal Socket (/dev/log).1756client # [ 1.799992] systemd[1]: Reached target Path Units.1757lb # [ 1.792919] systemd[1]: Created slice Slice /system/modprobe.1758client # [ 1.800012] systemd[1]: Reached target Slice Units.1759worker2 # [ 1.795602] systemd[1]: Listening on Journal Sockets.1760client # [ 1.800032] systemd[1]: Reached target Swaps.1761worker2 # [ 1.796631] systemd[1]: Listening on udev Control Socket.1762lb # [ 1.794337] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1763client # [ 1.800053] systemd[1]: Reached target Timer Units.1764worker2 # [ 1.797668] systemd[1]: Listening on udev Kernel Socket.1765client # [ 1.800251] systemd[1]: Listening on D-Bus System Message Bus Socket.1766lb # [ 1.795953] systemd[1]: Expecting device /dev/disk/by-label/nixos...1767worker2 # [ 1.798580] systemd[1]: Reached target Socket Units.1768client # [ 1.800418] systemd[1]: Listening on Journal Socket (/dev/log).1769lb # [ 1.797152] systemd[1]: Reached target Path Units.1770client # [ 1.800571] systemd[1]: Listening on Journal Sockets.1771lb # [ 1.798069] systemd[1]: Reached target Slice Units.1772client # [ 1.800714] systemd[1]: Listening on udev Control Socket.1773lb # [ 1.799098] systemd[1]: Reached target Swaps.1774worker2 # [ 1.801009] systemd[1]: Starting Create List of Static Device Nodes...1775client # [ 1.800839] systemd[1]: Listening on udev Kernel Socket.1776worker1 # [ 1.828832] systemd[1]: Starting Load Kernel Modules...1777lb # [ 1.799957] systemd[1]: Reached target Timer Units.1778client # [ 1.800864] systemd[1]: Reached target Socket Units.1779worker2 # [ 1.802284] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1780lb # [ 1.801082] systemd[1]: Listening on D-Bus System Message Bus Socket.1781worker1 # [ 1.829978] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1782client # [ 1.813442] systemd[1]: Starting Create List of Static Device Nodes...1783client # [ 1.814599] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1784lb # [ 1.802546] systemd[1]: Listening on Journal Socket (/dev/log).1785lb # [ 1.802716] systemd[1]: Listening on Journal Sockets.1786lb # [ 1.802871] systemd[1]: Listening on udev Control Socket.1787lb # [ 1.803003] systemd[1]: Listening on udev Kernel Socket.1788lb # [ 1.803031] systemd[1]: Reached target Socket Units.1789lb # [ 1.809773] systemd[1]: Starting Create List of Static Device Nodes...1790worker2 # [ 1.811684] systemd[1]: Mounting Kernel Configuration File System...1791lb # [ 1.811160] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1792client # [ 1.825890] systemd[1]: Mounting Kernel Configuration File System...1793worker1 # [ 1.842612] systemd[1]: Starting Coldplug All udev Devices...1794worker2 # [ 1.819820] systemd[1]: Starting Journal Service...1795lb # [ 1.819319] systemd[1]: Mounting Kernel Configuration File System...1796lb # [ 1.829001] systemd[1]: Starting Journal Service...1797worker1 # [ 1.861045] systemd[1]: Finished Create List of Static Device Nodes.1798worker1 # [ 1.862043] systemd[1]: Mounted Kernel Configuration File System.1799worker2 # [ 1.843941] systemd[1]: Starting Load Kernel Modules...1800worker2 # [ 1.844990] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1801client # [ 1.855456] systemd[1]: Starting Journal Service...1802lb # [ 1.846656] systemd[1]: Starting Load Kernel Modules...1803client # [ 1.858315] systemd[1]: Starting Load Kernel Modules...1804worker1 # [ 1.877344] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1805client # [ 1.858433] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1806lb # [ 1.847540] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1807lb # [ 1.854514] systemd[1]: Starting Coldplug All udev Devices...1808worker1 # [ 1.884831] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1809worker1 # [ 1.887554] systemd-journald[72]: Collecting audit messages is disabled.1810client # [ 1.870683] systemd[1]: Starting Coldplug All udev Devices...1811worker2 # [ 1.866628] systemd-journald[72]: Collecting audit messages is disabled.1812client # [ 1.882824] systemd[1]: Finished Create List of Static Device Nodes.1813client # [ 1.884500] systemd[1]: Mounted Kernel Configuration File System.1814worker1 # [ 1.905215] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1815lb # [ 1.875248] systemd[1]: Finished Create List of Static Device Nodes.1816worker2 # [ 1.876064] systemd[1]: Starting Coldplug All udev Devices...1817worker1 # [ 1.914565] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01818worker1 # [ 1.914802] [drm] features: -virgl +edid -resource_blob -host_visible1819worker1 # [ 1.914811] [drm] features: -context_init1820worker1 # [ 1.915515] [drm] number of scanouts: 11821worker1 # [ 1.915533] [drm] number of cap sets: 01822client # [ 1.904964] systemd-journald[72]: Collecting audit messages is disabled.1823lb # [ 1.890595] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1824worker2 # [ 1.891685] systemd[1]: Finished Create List of Static Device Nodes.1825lb # [ 1.890915] systemd[1]: Mounted Kernel Configuration File System.1826worker2 # [ 1.892666] systemd[1]: Mounted Kernel Configuration File System.1827client # [ 1.906149] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1828worker2 # [ 1.897520] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1829worker1 # [ 1.929533] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1830worker1 # [ 1.933292] systemd[1]: Starting Create Static Device Nodes in /dev...1831lb # [ 1.910230] systemd-journald[72]: Collecting audit messages is disabled.1832worker1 # [ 1.941423] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1833worker1 # [ 1.941442] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01834worker2 # [ 1.925481] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1835worker1 # [ 1.961179] Console: switching to colour frame buffer device 160x501836lb # [ 1.939894] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1837worker2 # [ 1.944153] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1838worker2 # [ 1.946385] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1839lb # [ 1.944494] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1840client # [ 1.951312] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1841client # [ 1.952920] systemd[1]: Starting Create Static Device Nodes in /dev...1842worker2 # [ 1.948877] systemd[1]: Starting Create Static Device Nodes in /dev...1843lb # [ 1.947067] systemd[1]: Starting Create Static Device Nodes in /dev...1844client # [ 1.968759] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1845worker1 # [ 1.985673] systemd[1]: Finished Create Static Device Nodes in /dev.1846lb # [ 1.958646] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1847worker1 # [ 1.985859] systemd[1]: Reached target Preparation for Local File Systems.1848worker1 # [ 1.985883] systemd[1]: Reached target Local File Systems.1849worker2 # [ 1.960920] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01850worker2 # [ 1.961159] [drm] features: -virgl +edid -resource_blob -host_visible1851worker1 # [ 1.990051] systemd[1]: Starting Rule-based Manager for Device Events and Files...1852worker2 # [ 1.961168] [drm] features: -context_init1853worker2 # [ 1.961871] [drm] number of scanouts: 11854worker2 # [ 1.961889] [drm] number of cap sets: 01855client # [ 1.974742] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1856lb # [ 1.967351] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01857lb # [ 1.967588] [drm] features: -virgl +edid -resource_blob -host_visible1858lb # [ 1.967599] [drm] features: -context_init1859lb # [ 1.968310] [drm] number of scanouts: 11860lb # [ 1.968327] [drm] number of cap sets: 01861worker1 # [ 2.001524] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1862client # [ 1.981802] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01863client # [ 1.982048] [drm] features: -virgl +edid -resource_blob -host_visible1864client # [ 1.982058] [drm] features: -context_init1865worker2 # [ 1.979877] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1866worker2 # [ 1.979893] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01867client # [ 2.006906] [drm] number of scanouts: 11868client # [ 2.006942] [drm] number of cap sets: 01869lb # [ 1.990609] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1870lb # [ 1.990628] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01871worker1 # [ 2.025153] systemd[1]: Finished Load Kernel Modules.1872worker2 # [ 2.000089] systemd[1]: Finished Create Static Device Nodes in /dev.1873worker2 # [ 2.000277] systemd[1]: Reached target Preparation for Local File Systems.1874worker2 # [ 2.000306] systemd[1]: Reached target Local File Systems.1875worker1 # [ 2.029293] systemd[1]: Starting Apply Kernel Variables...1876client # [ 2.011391] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1877client # [ 2.011412] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01878client # [ 2.020158] systemd[1]: Finished Create Static Device Nodes in /dev.1879client # [ 2.020546] systemd[1]: Reached target Preparation for Local File Systems.1880client # [ 2.020576] systemd[1]: Reached target Local File Systems.1881worker2 # [ 2.007909] systemd[1]: Starting Rule-based Manager for Device Events and Files...1882lb # [ 2.014843] systemd[1]: Finished Create Static Device Nodes in /dev.1883lb # [ 2.015069] systemd[1]: Reached target Preparation for Local File Systems.1884lb # [ 2.015097] systemd[1]: Reached target Local File Systems.1885lb # [ 2.018966] systemd[1]: Starting Rule-based Manager for Device Events and Files...1886worker2 # [ 2.015449] Console: switching to colour frame buffer device 160x501887client # [ 2.031160] systemd[1]: Starting Rule-based Manager for Device Events and Files...1888lb # [ 2.021581] Console: switching to colour frame buffer device 160x501889worker1 # [ 2.046948] systemd-modules-load[73]: Inserted module 'dm_mod'1890client # [ 2.039814] Console: switching to colour frame buffer device 160x501891worker1 # [ 2.065188] systemd[1]: Started Journal Service.1892worker2 # [ 2.040293] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1893worker1 # [ 2.064305] systemd-modules-load[73]: Module 'virtio_balloon' is built in1894lb # [ 2.050868] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1895client # [ 2.063284] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1896worker2 # [ 2.056228] systemd[1]: Finished Load Kernel Modules.1897worker2 # [ 2.059634] systemd[1]: Starting Apply Kernel Variables...1898worker1 # [ 2.072148] systemd-modules-load[73]: Module 'virtio_console' is built in1899client # [ 2.072239] systemd[1]: Finished Load Kernel Modules.1900lb # [ 2.062816] systemd[1]: Finished Load Kernel Modules.1901worker1 # [ 2.077328] systemd-modules-load[73]: Inserted module 'virtio_gpu'1902worker1 # [ 2.079509] systemd-modules-load[73]: Module 'virtio_rng' is built in1903client # [ 2.079213] systemd[1]: Starting Apply Kernel Variables...1904worker2 # [ 2.055196] systemd-modules-load[74]: Inserted module 'dm_mod'1905worker2 # [ 2.056389] systemd-modules-load[74]: Module 'virtio_balloon' is built in1906lb # [ 2.069372] systemd[1]: Starting Apply Kernel Variables...1907worker2 # [ 2.075749] systemd[1]: Started Journal Service.1908worker1 # [ 2.092418] systemd[1]: Starting Create System Files and Directories...1909worker1 # [ 2.093671] systemd[1]: Finished Apply Kernel Variables.1910worker1 # [ 2.094819] systemd-udevd[79]: Using default interface naming scheme 'v261'.1911worker2 # [ 2.069292] systemd-modules-load[74]: Module 'virtio_console' is built in1912worker2 # [ 2.071476] systemd-modules-load[74]: Inserted module 'virtio_gpu'1913client # [ 2.080723] systemd-modules-load[73]: Inserted module 'dm_mod'1914client # [ 2.100540] systemd[1]: Started Journal Service.1915client # [ 2.085296] systemd-modules-load[73]: Module 'virtio_balloon' is built in1916client # [ 2.086433] systemd-modules-load[73]: Module 'virtio_console' is built in1917client # [ 2.087504] systemd-modules-load[73]: Inserted module 'virtio_gpu'1918worker2 # [ 2.084441] systemd-modules-load[74]: Module 'virtio_rng' is built in1919worker2 # [ 2.085529] systemd[1]: Starting Create System Files and Directories...1920lb # [ 2.086148] systemd-modules-load[73]: Inserted module 'dm_mod'1921worker1 # [ 2.114636] systemd[1]: Finished Create System Files and Directories.1922lb # [ 2.102346] systemd[1]: Started Journal Service.1923lb # [ 2.091223] systemd-modules-load[73]: Module 'virtio_balloon' is built in1924client # [ 2.100497] systemd-modules-load[73]: Module 'virtio_rng' is built in1925lb # [ 2.096496] systemd-modules-load[73]: Module 'virtio_console' is built in1926lb # [ 2.097629] systemd-modules-load[73]: Inserted module 'virtio_gpu'1927lb # [ 2.098632] systemd-modules-load[73]: Module 'virtio_rng' is built in1928client # [ 2.108145] systemd[1]: Starting Create System Files and Directories...1929worker2 # [ 2.102610] systemd[1]: Finished Apply Kernel Variables.1930worker1 # [ 2.131328] systemd[1]: Started Rule-based Manager for Device Events and Files.1931worker2 # [ 2.117253] systemd-udevd[78]: Using default interface naming scheme 'v261'.1932lb # [ 2.117008] systemd[1]: Starting Create System Files and Directories...1933lb # [ 2.119065] systemd-udevd[79]: Using default interface naming scheme 'v261'.1934client # [ 2.131511] systemd-udevd[79]: Using default interface naming scheme 'v261'.1935worker2 # [ 2.128471] systemd[1]: Finished Create System Files and Directories.1936lb # [ 2.132069] systemd[1]: Finished Apply Kernel Variables.1937client # [ 2.144591] systemd[1]: Finished Apply Kernel Variables.1938lb # [ 2.142748] systemd[1]: Finished Create System Files and Directories.1939worker2 # [ 2.150083] systemd[1]: Started Rule-based Manager for Device Events and Files.1940client # [ 2.160944] systemd[1]: Finished Create System Files and Directories.1941worker1 # [ 2.192663] systemd[1]: Starting Virtual Console Setup...1942lb # [ 2.167352] systemd[1]: Started Rule-based Manager for Device Events and Files.1943client # [ 2.177855] systemd[1]: Started Rule-based Manager for Device Events and Files.1944worker2 # [ 2.203719] systemd[1]: Starting Virtual Console Setup...1945worker1 # [ 2.244521] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1946worker1 # [ 2.248003] systemd[1]: Finished Virtual Console Setup.1947client # [ 2.236135] systemd[1]: Starting Virtual Console Setup...1948lb # [ 2.232404] systemd[1]: Starting Virtual Console Setup...1949worker2 # [ 2.256467] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1950worker2 # [ 2.259956] systemd[1]: Finished Virtual Console Setup.1951lb # [ 2.276491] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1952client # [ 2.284476] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1953lb # [ 2.279140] systemd[1]: Finished Virtual Console Setup.1954client # [ 2.287744] systemd[1]: Finished Virtual Console Setup.1955worker1 # [ 2.865099] systemd[1]: Finished Coldplug All udev Devices.1956worker1 # [ 2.867060] systemd[1]: Reached target System Initialization.1957worker1 # [ 2.872804] systemd[1]: Reached target Basic System.1958worker2 # [ 2.861101] systemd[1]: Finished Coldplug All udev Devices.1959worker2 # [ 2.863145] systemd[1]: Reached target System Initialization.1960worker2 # [ 2.869099] systemd[1]: Reached target Basic System.1961client # [ 2.893600] systemd[1]: Finished Coldplug All udev Devices.1962client # [ 2.896155] systemd[1]: Reached target System Initialization.1963client # [ 2.897061] systemd[1]: Reached target Basic System.1964lb # [ 2.900178] systemd[1]: Finished Coldplug All udev Devices.1965lb # [ 2.902066] systemd[1]: Reached target System Initialization.1966lb # [ 2.902934] systemd[1]: Reached target Basic System.1967worker1 # [ 3.015615] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1968worker1 # [ 3.036752] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1969worker2 # [ 3.015076] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1970worker1 # [ 3.052086] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1971client # [ 3.040538] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972worker2 # [ 3.040312] (udev-worker)[104]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973worker2 # [ 3.044541] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1974lb # [ 3.059464] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975lb # [ 3.065696] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1976client # [ 3.078261] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1977lb # [ 3.077843] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1978client # [ 3.086392] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1979worker1 # [ 3.109915] systemd[1]: Found device /dev/disk/by-label/nixos.1980worker1 # [ 3.112141] systemd[1]: Reached target Initrd Root Device.1981worker1 # [ 3.116490] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982worker2 # [ 3.100752] systemd[1]: Found device /dev/disk/by-label/nixos.1983worker2 # [ 3.105489] systemd[1]: Reached target Initrd Root Device.1984worker2 # [ 3.110667] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985worker1 # [ 3.165824] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1986worker1 # [ 3.176164] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1987worker1 # [ 3.178689] systemd[1]: Mounting /sysroot...1988lb # [ 3.156133] systemd[1]: Found device /dev/disk/by-label/nixos.1989lb # [ 3.161060] systemd[1]: Reached target Initrd Root Device.1990client # [ 3.172885] systemd[1]: Found device /dev/disk/by-label/nixos.1991worker2 # [ 3.167976] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1992lb # [ 3.168106] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1993client # [ 3.178629] systemd[1]: Reached target Initrd Root Device.1994worker2 # [ 3.173844] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995client # [ 3.181956] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1996worker2 # [ 3.179837] systemd[1]: Mounting /sysroot...1997worker1 # [ 3.228807] EXT4-fs (vda): mounted filesystem 33133b99-1616-4b97-8015-09ee09300e84 r/w with ordered data mode. Quota mode: none.1998worker1 # [ 3.219604] systemd[1]: Mounted /sysroot.1999worker1 # [ 3.221528] systemd[1]: Reached target Initrd Root File System.2000worker1 # [ 3.228138] systemd[1]: Starting Mountpoints Configured in the Real Root...
loading 2,177 hidden lines…
4178client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24179worker2 # [ 42.470235] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:603484180client # don't know how to build these paths:4181client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4182worker2 # [ 42.480433] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484183worker2 # [ 42.509079] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14184client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.29 seconds)4185client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4186client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.14 seconds)4187lb: must succeed: curl -sf localhost:9901/clusters4188lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4189lb: must succeed: curl -sf localhost:9901/clusters4190lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4191worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4192worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4193(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.15 seconds)4194subtest: inputs already in the cache are not sent to the scheduler4195lb: must succeed: curl -sf localhost:9901/clusters4196lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4197lb: must succeed: curl -sf localhost:9901/clusters4198lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4199worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4200worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4201client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24202worker2 # [ 43.328321] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:603484203client # don't know how to build these paths:4204client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4205worker2 # [ 43.338355] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484206worker2 # [ 43.354066] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04207worker2 # [ 43.360303] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14208worker1 # [ 43.429797] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764209worker1 # [ 43.442346] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416764210worker1 # [ 43.452955] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.055Z level=INFO msg="All 2 paths already cached"4211client # copying 1 paths to 'lb:50051'...4212client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4213worker1 # [ 43.464661] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416764214worker1 # [ 43.479229] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4215worker1 # [ 43.641135] nix-daemon[829]: accepted connection from pid 975, user nix-grpc-daemon (trusted)4216worker1 # [ 43.658311] nix-daemon[829]: reaped child process 978, status = succeeded4217worker1 # [ 43.759984] nix-daemon[829]: accepted connection from pid 980, user nix-grpc-daemon (trusted)4218worker1 # [ 43.778883] nix-daemon[829]: reaped child process 983, status = succeeded4219lb # [ 43.754785] niks3-server[937]: 2026/09/22 15:45:56 INFO Received uploads request method=POST path=/api/pending_closures4220worker1 # [ 43.790688] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.393Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4221worker1 # [ 43.792217] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.393Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4222lb # [ 43.784958] niks3-server[937]: 2026/09/22 15:45:56 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4223worker1 # [ 43.824765] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.427Z level=INFO msg="Uploading 1 narinfos"4224lb # [ 43.799955] niks3-server[937]: 2026/09/22 15:45:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4225lb # [ 43.810908] niks3-server[937]: 2026/09/22 15:45:56 INFO Signed narinfos id=7 count=14226lb # [ 43.817628] niks3-server[937]: 2026/09/22 15:45:56 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4227worker1 # [ 43.857315] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.460Z level=INFO msg="Upload complete. (360ms)"4228lb # [ 43.832726] niks3-server[937]: 2026/09/22 15:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4229lb # [ 43.838433] niks3-server[937]: 2026/09/22 15:45:56 INFO Completed upload id=74230lb # [ 43.841766] niks3-server[937]: 2026/09/22 15:45:56 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4231worker1 # [ 43.868061] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=7524232worker1 # [ 43.877015] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764233worker1 # [ 43.884756] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4234worker1 # [ 43.889253] nix-daemon[829]: reaped child process 973, status = succeeded4235client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4236worker1 # [ 43.901017] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4237client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4238client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4239client # farm-top> LOG-farm-top-t1b4240worker1 # [ 44.999454] nix-daemon[829]: reaped child process 987, status = succeeded4241worker1 # [ 45.075814] nix-daemon[829]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)4242worker1 # [ 45.096506] nix-daemon[829]: reaped child process 1007, status = succeeded4243lb # [ 45.077993] niks3-server[937]: 2026/09/22 15:45:57 INFO Received uploads request method=POST path=/api/pending_closures4244worker1 # [ 45.112404] nix-grpc-daemon[842]: time=2026-09-22T15:45:57.714Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4245worker1 # [ 45.117158] nix-grpc-daemon[842]: time=2026-09-22T15:45:57.717Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4246lb # [ 45.117040] niks3-server[937]: 2026/09/22 15:45:57 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4247lb # [ 45.131214] niks3-server[937]: 2026/09/22 15:45:57 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4248lb # [ 45.139830] niks3-server[937]: 2026/09/22 15:45:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4249worker1 # [ 45.171901] nix-grpc-daemon[842]: time=2026-09-22T15:45:57.774Z level=INFO msg="Uploading 1 narinfos"4250lb # [ 45.149990] niks3-server[937]: 2026/09/22 15:45:57 INFO Signed narinfos id=8 count=14251lb # [ 45.155583] niks3-server[937]: 2026/09/22 15:45:57 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4252lb # [ 45.166980] niks3-server[937]: 2026/09/22 15:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4253lb # [ 45.173949] niks3-server[937]: 2026/09/22 15:45:57 INFO Completed upload id=84254worker1 # [ 45.200130] nix-grpc-daemon[842]: time=2026-09-22T15:45:57.802Z level=INFO msg="Upload complete. (289ms)"4255lb # [ 45.176721] niks3-server[937]: 2026/09/22 15:45:57 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4256worker1 # [ 45.213239] nix-grpc-daemon[765]: ts=2026-09-22T15:45:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14257worker1 # [ 45.231535] nix-daemon[829]: reaped child process 985, status = succeeded4258client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.11 seconds)4259client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4260client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4261lb: must succeed: curl -sf localhost:9901/clusters4262lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4263lb: must succeed: curl -sf localhost:9901/clusters4264lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4265worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4266worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4267(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.63 seconds)4268subtest: outputs read back through any worker carry the cache signature4269client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24270worker2 # [ 45.810160] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:603484271client # don't know how to build these paths:4272client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4273worker2 # [ 45.822431] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484274worker2 # [ 45.846451] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04275worker2 # [ 45.857020] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14276worker2 # [ 45.866014] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484277worker2 # [ 45.873482] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484278client # copying 1 paths to 'lb:50051'...4279client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4280worker2 # [ 45.886170] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:603484281worker2 # [ 45.901848] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4282worker2 # [ 45.927252] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=info event=niks3_push_started pid=9104283worker2 # [ 46.093160] nix-grpc-daemon[910]: time=2026-09-22T15:45:58.690Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4284worker2 # [ 46.201987] nix-daemon[856]: accepted connection from pid 922, user nix-grpc-daemon (trusted)4285worker2 # [ 46.223665] nix-daemon[856]: reaped child process 925, status = succeeded4286worker2 # [ 46.325638] nix-daemon[856]: accepted connection from pid 927, user nix-grpc-daemon (trusted)4287worker2 # [ 46.344879] nix-daemon[856]: reaped child process 930, status = succeeded4288lb # [ 46.343007] niks3-server[937]: 2026/09/22 15:45:58 INFO Received uploads request method=POST path=/api/pending_closures4289worker2 # [ 46.354761] nix-grpc-daemon[910]: time=2026-09-22T15:45:58.952Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4290worker2 # [ 46.356754] nix-grpc-daemon[910]: time=2026-09-22T15:45:58.953Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4291lb # [ 46.378286] niks3-server[937]: 2026/09/22 15:45:58 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4292worker2 # [ 46.392078] nix-grpc-daemon[910]: time=2026-09-22T15:45:58.991Z level=INFO msg="Uploading 1 narinfos"4293lb # [ 46.392647] niks3-server[937]: 2026/09/22 15:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4294lb # [ 46.404509] niks3-server[937]: 2026/09/22 15:45:58 INFO Signed narinfos id=9 count=14295lb # [ 46.410488] niks3-server[937]: 2026/09/22 15:45:58 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4296worker2 # [ 46.420083] nix-grpc-daemon[910]: time=2026-09-22T15:45:59.019Z level=INFO msg="Upload complete. (327ms)"4297lb # [ 46.420189] niks3-server[937]: 2026/09/22 15:45:59 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4298worker2 # [ 46.426300] nix-grpc-daemon[772]: ts=2026-09-22T15:45:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=5124299lb # [ 46.424889] niks3-server[937]: 2026/09/22 15:45:59 INFO Completed upload id=94300lb # [ 46.431642] niks3-server[937]: 2026/09/22 15:45:59 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4301worker2 # [ 46.438090] nix-grpc-daemon[772]: ts=2026-09-22T15:45:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484302worker2 # [ 46.443950] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4303worker2 # [ 46.448376] nix-daemon[856]: reaped child process 908, status = succeeded4304client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4305worker2 # [ 46.460200] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4306client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4307client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4308client # farm-a> LOG-farm-a-sig4309worker2 # [ 47.540809] nix-daemon[856]: reaped child process 934, status = succeeded4310worker2 # [ 47.645355] nix-daemon[856]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4311worker2 # [ 47.664284] nix-daemon[856]: reaped child process 953, status = succeeded4312lb # [ 47.670833] niks3-server[937]: 2026/09/22 15:46:00 INFO Received uploads request method=POST path=/api/pending_closures4313worker2 # [ 47.684426] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.280Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4314worker2 # [ 47.686214] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.280Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4315lb # [ 47.712530] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4316worker2 # [ 47.729837] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.327Z level=INFO msg="Uploading 1 narinfos"4317lb # [ 47.728671] niks3-server[937]: 2026/09/22 15:46:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4318lb # [ 47.739006] niks3-server[937]: 2026/09/22 15:46:00 INFO Signed narinfos id=10 count=14319lb # [ 47.745361] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4320lb # [ 47.752135] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4321worker2 # [ 47.761424] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.360Z level=INFO msg="Upload complete. (291ms)"4322lb # [ 47.763589] niks3-server[937]: 2026/09/22 15:46:00 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4323lb # [ 47.770993] niks3-server[937]: 2026/09/22 15:46:00 INFO Completed upload id=104324lb # [ 47.773837] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4325worker2 # [ 47.777361] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14326worker2 # [ 47.799710] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04327worker2 # [ 47.804661] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14328worker2 # [ 47.808995] nix-daemon[856]: reaped child process 932, status = succeeded4329worker2 # [ 47.837720] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484330worker2 # [ 47.853836] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484331worker2 # [ 47.867929] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.467Z level=INFO msg="All 1 paths already cached"4332client # copying 1 paths to 'lb:50051'...4333client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4334worker2 # [ 47.879063] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:603484335worker2 # [ 47.893241] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4336worker2 # [ 48.052122] nix-daemon[856]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4337worker2 # [ 48.074451] nix-daemon[856]: reaped child process 962, status = succeeded4338worker2 # [ 48.177656] nix-daemon[856]: accepted connection from pid 964, user nix-grpc-daemon (trusted)4339worker2 # [ 48.198518] nix-daemon[856]: reaped child process 967, status = succeeded4340lb # [ 48.196343] niks3-server[937]: 2026/09/22 15:46:00 INFO Received uploads request method=POST path=/api/pending_closures4341worker2 # [ 48.207909] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.806Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4342worker2 # [ 48.209762] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.806Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4343lb # [ 48.225735] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4344worker2 # [ 48.249604] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.848Z level=INFO msg="Uploading 1 narinfos"4345lb # [ 48.241189] niks3-server[937]: 2026/09/22 15:46:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4346lb # [ 48.254084] niks3-server[937]: 2026/09/22 15:46:00 INFO Signed narinfos id=11 count=14347lb # [ 48.262292] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4348lb # [ 48.272499] niks3-server[937]: 2026/09/22 15:46:00 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4349worker2 # [ 48.282384] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.881Z level=INFO msg="Upload complete. (372ms)"4350lb # [ 48.284374] niks3-server[937]: 2026/09/22 15:46:00 INFO Completed upload id=114351lb # [ 48.288070] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4352worker2 # [ 48.292355] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=6244353worker2 # [ 48.301123] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484354worker2 # [ 48.313455] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4355worker2 # [ 48.319166] nix-daemon[856]: reaped child process 957, status = succeeded4356client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4357worker2 # [ 48.329910] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4358client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4359lb # [ 48.538230] envoy[894]: conn peer=[2001:db8:1::1]:38300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=4382 ms=59694360lb # [ 48.543511] envoy[894]: conn peer=[2001:db8:1::1]:55730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=624361lb # [ 48.548389] envoy[894]: conn peer=[2001:db8:1::1]:55746 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1914 ms=18704362client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4363client # farm-b> LOG-farm-b-sig4364worker2 # [ 49.219811] nix-daemon[856]: reaped child process 971, status = succeeded4365worker2 # [ 49.284257] nix-daemon[856]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4366worker2 # [ 49.301681] nix-daemon[856]: reaped child process 990, status = succeeded4367lb # [ 49.309140] niks3-server[937]: 2026/09/22 15:46:01 INFO Received uploads request method=POST path=/api/pending_closures4368worker2 # [ 49.319710] nix-grpc-daemon[910]: time=2026-09-22T15:46:01.918Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4369worker2 # [ 49.321321] nix-grpc-daemon[910]: time=2026-09-22T15:46:01.918Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4370lb # [ 49.340250] niks3-server[937]: 2026/09/22 15:46:01 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4371worker2 # [ 49.360643] nix-grpc-daemon[910]: time=2026-09-22T15:46:01.959Z level=INFO msg="Uploading 1 narinfos"4372lb # [ 49.359743] niks3-server[937]: 2026/09/22 15:46:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4373lb # [ 49.371515] niks3-server[937]: 2026/09/22 15:46:01 INFO Signed narinfos id=12 count=14374lb # [ 49.378581] niks3-server[937]: 2026/09/22 15:46:01 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4375lb # [ 49.391155] niks3-server[937]: 2026/09/22 15:46:01 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4376worker2 # [ 49.398090] nix-grpc-daemon[910]: time=2026-09-22T15:46:01.997Z level=INFO msg="Upload complete. (256ms)"4377lb # [ 49.398802] niks3-server[937]: 2026/09/22 15:46:01 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4378lb # [ 49.403642] niks3-server[937]: 2026/09/22 15:46:01 INFO Completed upload id=124379lb # [ 49.407635] niks3-server[937]: 2026/09/22 15:46:01 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4380worker2 # [ 49.416388] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14381worker2 # [ 49.434416] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04382worker2 # [ 49.438933] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14383worker2 # [ 49.443826] nix-daemon[856]: reaped child process 969, status = succeeded4384worker2 # [ 49.473315] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484385worker2 # [ 49.488532] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484386worker2 # [ 49.502448] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.101Z level=INFO msg="All 2 paths already cached"4387client # copying 1 paths to 'lb:50051'...4388client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4389worker2 # [ 49.510124] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:603484390worker2 # [ 49.524124] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4391worker2 # [ 49.680046] nix-daemon[856]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4392worker2 # [ 49.700155] nix-daemon[856]: reaped child process 999, status = succeeded4393worker2 # [ 49.800873] nix-daemon[856]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4394worker2 # [ 49.820673] nix-daemon[856]: reaped child process 1004, status = succeeded4395lb # [ 49.821483] niks3-server[937]: 2026/09/22 15:46:02 INFO Received uploads request method=POST path=/api/pending_closures4396worker2 # [ 49.830730] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.429Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4397worker2 # [ 49.832404] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.429Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4398lb # [ 49.849244] niks3-server[937]: 2026/09/22 15:46:02 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4399lb # [ 49.864233] niks3-server[937]: 2026/09/22 15:46:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4400worker2 # [ 49.870722] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.469Z level=INFO msg="Uploading 1 narinfos"4401lb # [ 49.876859] niks3-server[937]: 2026/09/22 15:46:02 INFO Signed narinfos id=13 count=14402lb # [ 49.879300] niks3-server[937]: 2026/09/22 15:46:02 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4403lb # [ 49.892171] niks3-server[937]: 2026/09/22 15:46:02 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4404lb # [ 49.899070] niks3-server[937]: 2026/09/22 15:46:02 INFO Completed upload id=134405worker2 # [ 49.899641] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.498Z level=INFO msg="Upload complete. (359ms)"4406lb # [ 49.901956] niks3-server[937]: 2026/09/22 15:46:02 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4407worker2 # [ 49.907678] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=7524408worker2 # [ 49.916925] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484409worker2 # [ 49.929794] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4410worker2 # [ 49.934502] nix-daemon[856]: reaped child process 994, status = succeeded4411client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4412worker2 # [ 49.948172] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4413client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4414client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4415client # farm-top> LOG-farm-top-sig4416worker2 # [ 51.028339] nix-daemon[856]: reaped child process 1008, status = succeeded4417worker2 # [ 51.141020] nix-daemon[856]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4418worker2 # [ 51.162424] nix-daemon[856]: reaped child process 1027, status = succeeded4419lb # [ 51.174630] niks3-server[937]: 2026/09/22 15:46:03 INFO Received uploads request method=POST path=/api/pending_closures4420worker2 # [ 51.190953] nix-grpc-daemon[910]: time=2026-09-22T15:46:03.789Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4421worker2 # [ 51.194675] nix-grpc-daemon[910]: time=2026-09-22T15:46:03.789Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4422lb # [ 51.238119] niks3-server[937]: 2026/09/22 15:46:03 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4423lb # [ 51.250631] niks3-server[937]: 2026/09/22 15:46:03 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4424lb # [ 51.264188] niks3-server[937]: 2026/09/22 15:46:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4425worker2 # [ 51.273319] nix-grpc-daemon[910]: time=2026-09-22T15:46:03.872Z level=INFO msg="Uploading 1 narinfos"4426lb # [ 51.275390] niks3-server[937]: 2026/09/22 15:46:03 INFO Signed narinfos id=14 count=14427lb # [ 51.281557] niks3-server[937]: 2026/09/22 15:46:03 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4428lb # [ 51.295657] niks3-server[937]: 2026/09/22 15:46:03 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4429lb # [ 51.304202] niks3-server[937]: 2026/09/22 15:46:03 INFO Completed upload id=144430worker2 # [ 51.304984] nix-grpc-daemon[910]: time=2026-09-22T15:46:03.904Z level=INFO msg="Upload complete. (344ms)"4431lb # [ 51.307299] niks3-server[937]: 2026/09/22 15:46:03 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4432worker2 # [ 51.321506] nix-grpc-daemon[772]: ts=2026-09-22T15:46:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14433worker2 # [ 51.343667] nix-daemon[856]: reaped child process 1006, status = succeeded4434client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.79 seconds)4435client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4436client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4437client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4438worker1 # [ 51.750505] nix-grpc-daemon[765]: ts=2026-09-22T15:46:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:343924439client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4440client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4441client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4442client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4443worker1 # [ 51.809453] nix-grpc-daemon[765]: ts=2026-09-22T15:46:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:343924444worker1 # [ 51.820155] nix-grpc-daemon[765]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4445worker1 # [ 51.843180] nix-grpc-daemon[765]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4446worker1 # [ 51.863894] nix-grpc-daemon[765]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4447client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.33 seconds)4448client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4449worker2 # [ 52.083360] nix-grpc-daemon[772]: ts=2026-09-22T15:46:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:462224450worker2 # [ 52.127191] nix-grpc-daemon[772]: ts=2026-09-22T15:46:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:462224451client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.30 seconds)4452client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4453client # finding garbage collector roots...4454client # removing stale temporary roots file "/nix/var/nix/temproots/857"4455client # removing stale temporary roots file "/nix/var/nix/temproots/843"4456client # removing stale temporary roots file "/nix/var/nix/temproots/823"4457client # removing stale temporary roots file "/nix/var/nix/temproots/785"4458client # removing stale temporary roots file "/nix/var/nix/temproots/865"4459client # removing stale temporary roots file "/nix/var/nix/temproots/880"4460client # removing stale temporary roots file "/nix/var/nix/temproots/801"4461client # removing stale temporary roots file "/nix/var/nix/temproots/835"4462client # deleting specified paths...4463client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4464client # deleting unused links...4465client # note: hard linking is currently saving 0.0 KiB4466worker2 # [ 52.555618] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:603484467worker2 # [ 52.567221] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484468worker2 # [ 52.584482] nix-grpc-daemon[910]: time=2026-09-22T15:46:05.182Z level=INFO msg="All 1 paths already cached"4469worker1 # [ 52.621840] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:416764470worker2 # [ 52.608936] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:603484471client # copying 3 paths...4472worker1 # [ 52.655047] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:416764473client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4474worker2 # [ 52.667036] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:603484475worker2 # [ 52.675597] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_out=2324476worker1 # [ 52.716934] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:416764477worker1 # [ 52.725441] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=1764478worker1 # [ 52.737051] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:416764479worker1 # [ 52.743980] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=1284480client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4481client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4482client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.62 seconds)4483(finished: subtest: outputs read back through any worker carry the cache signature, in 7.24 seconds)4484subtest: two clients wanting the same drv build it once4485client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4486client # [ 52.885845] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4487client # Running as unit: dedup1.service; invocation ID: 15138f67272945038baf1659223cce824488client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4489client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4490client # [ 53.088158] nix[932]: don't know how to build these paths:4491client # [ 53.090030] nix[932]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4492worker1 # [ 53.106233] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:416764493worker2 # [ 53.098301] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04494worker2 # [ 53.105081] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14495worker1 # [ 53.171624] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764496worker1 # [ 53.193870] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416764497client # [ 53.193438] nix[932]: copying 1 paths to 'lb:50051'...4498worker1 # [ 53.214168] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416764499client # [ 53.199623] nix[932]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4500worker1 # [ 53.231231] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4501worker1 # [ 53.421329] nix-daemon[829]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4502worker1 # [ 53.443851] nix-daemon[829]: reaped child process 1017, status = succeeded4503worker1 # [ 53.557515] nix-daemon[829]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4504worker1 # [ 53.579499] nix-daemon[829]: reaped child process 1022, status = succeeded4505lb # [ 53.551549] niks3-server[937]: 2026/09/22 15:46:06 INFO Received uploads request method=POST path=/api/pending_closures4506worker1 # [ 53.593368] nix-grpc-daemon[842]: time=2026-09-22T15:46:06.195Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4507worker1 # [ 53.594859] nix-grpc-daemon[842]: time=2026-09-22T15:46:06.195Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4508lb # [ 53.613074] niks3-server[937]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4509lb # [ 53.637330] niks3-server[937]: 2026/09/22 15:46:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4510worker1 # [ 53.670248] nix-grpc-daemon[842]: time=2026-09-22T15:46:06.272Z level=INFO msg="Uploading 1 narinfos"4511lb # [ 53.648382] niks3-server[937]: 2026/09/22 15:46:06 INFO Signed narinfos id=15 count=14512lb # [ 53.657688] niks3-server[937]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4513lb # [ 53.673798] niks3-server[937]: 2026/09/22 15:46:06 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4514lb # [ 53.680811] niks3-server[937]: 2026/09/22 15:46:06 INFO Completed upload id=154515worker1 # [ 53.706891] nix-grpc-daemon[842]: time=2026-09-22T15:46:06.309Z level=INFO msg="Upload complete. (452ms)"4516lb # [ 53.683470] niks3-server[937]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4517worker1 # [ 53.721011] nix-grpc-daemon[765]: ts=2026-09-22T15:46:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=4324518worker1 # [ 53.732818] nix-grpc-daemon[765]: ts=2026-09-22T15:46:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764519worker1 # [ 53.745771] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4520worker1 # [ 53.752235] nix-daemon[829]: reaped child process 1012, status = succeeded4521client # [ 53.745969] nix[932]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4522worker1 # [ 53.769089] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4523client # [ 53.979412] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4524client # Running as unit: dedup2.service; invocation ID: 6dcbc1eb68c543ef86b95edcc76edfd24525client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4526client # [ 54.217050] nix[948]: don't know how to build these paths:4527client # [ 54.219121] nix[948]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4528worker2 # [ 54.208972] nix-grpc-daemon[772]: ts=2026-09-22T15:46:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:603484529worker2 # [ 54.222913] nix-grpc-daemon[772]: ts=2026-09-22T15:46:06Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04530worker1 # [ 54.255642] nix-grpc-daemon[765]: ts=2026-09-22T15:46:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764531client # [ 54.244696] nix[948]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4532client: must succeed: sleep 24533client: (finished: must succeed: sleep 2, in 2.03 seconds)4534client: waiting for success: ! systemctl is-active dedup1 dedup24535worker1 # [ 57.678913] nix-daemon[829]: reaped child process 1026, status = succeeded4536worker1 # [ 57.750337] nix-daemon[829]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)4537worker1 # [ 57.771156] nix-daemon[829]: reaped child process 1059, status = succeeded4538lb # [ 57.746632] niks3-server[937]: 2026/09/22 15:46:10 INFO Received uploads request method=POST path=/api/pending_closures4539worker1 # [ 57.784511] nix-grpc-daemon[842]: time=2026-09-22T15:46:10.385Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4540worker1 # [ 57.786661] nix-grpc-daemon[842]: time=2026-09-22T15:46:10.385Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4541lb # [ 57.791232] niks3-server[937]: 2026/09/22 15:46:10 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4542lb # [ 57.803635] niks3-server[937]: 2026/09/22 15:46:10 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4543worker1 # [ 57.841179] nix-grpc-daemon[842]: time=2026-09-22T15:46:10.443Z level=INFO msg="Uploading 1 narinfos"4544lb # [ 57.814696] niks3-server[937]: 2026/09/22 15:46:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4545lb # [ 57.820302] niks3-server[937]: 2026/09/22 15:46:10 INFO Signed narinfos id=16 count=14546lb # [ 57.822438] niks3-server[937]: 2026/09/22 15:46:10 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4547lb # [ 57.833477] niks3-server[937]: 2026/09/22 15:46:10 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4548lb # [ 57.841412] niks3-server[937]: 2026/09/22 15:46:10 INFO Completed upload id=164549worker1 # [ 57.866475] nix-grpc-daemon[842]: time=2026-09-22T15:46:10.469Z level=INFO msg="Upload complete. (282ms)"4550lb # [ 57.846174] niks3-server[937]: 2026/09/22 15:46:10 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4551worker1 # [ 57.879135] nix-grpc-daemon[765]: ts=2026-09-22T15:46:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14552worker1 # [ 57.899092] nix-grpc-daemon[765]: ts=2026-09-22T15:46:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14553worker1 # [ 57.907371] nix-daemon[829]: reaped child process 1024, status = succeeded4554client # [ 57.898420] systemd[1]: dedup1.service: Deactivated successfully.4555client # [ 57.902850] systemd[1]: dedup1.service: Consumed 184ms CPU time over 5.016s wall clock time, 39.5M memory peak, 7.3K incoming IP traffic, 8.3K outgoing IP traffic.4556client # [ 57.950238] systemd[1]: dedup2.service: Deactivated successfully.4557client # [ 57.954150] systemd[1]: dedup2.service: Consumed 210ms CPU time over 3.974s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4558lb # [ 58.538314] envoy[894]: conn peer=[2001:db8:1::1]:55760 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9498 tx=4359 ms=55244559lb # [ 58.543288] envoy[894]: conn peer=[2001:db8:1::1]:54708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2914 ms=2484560client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4561lb # [ 58.548175] envoy[894]: conn peer=[2001:db8:1::1]:54742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=954562client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4563lb # [ 58.553237] envoy[894]: conn peer=[2001:db8:1::1]:54726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=1164564lb # [ 58.557978] envoy[894]: conn peer=[2001:db8:1::1]:54716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=1394565lb # [ 58.562945] envoy[894]: conn peer=[2001:db8:1::1]:54758 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1563 ms=47944566lb # [ 58.568746] envoy[894]: conn peer=[2001:db8:1::1]:54762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1198 ms=37164567client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4568worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4569worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4570worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4571worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4572worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4573worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4574worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4575worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4576(finished: subtest: two clients wanting the same drv build it once, in 6.04 seconds)4577subtest: --timeout and --max-silent-time reach the builder4578worker2 # [ 59.098862] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:603484579worker2 # [ 59.115885] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04580worker2 # [ 59.123102] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14581worker2 # [ 59.132874] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484582worker2 # [ 59.145150] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484583worker2 # [ 59.160525] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:603484584worker2 # [ 59.175850] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4585worker2 # [ 59.333330] nix-daemon[856]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)4586worker2 # [ 59.356093] nix-daemon[856]: reaped child process 1061, status = succeeded4587worker2 # [ 59.455819] nix-daemon[856]: accepted connection from pid 1063, user nix-grpc-daemon (trusted)4588worker2 # [ 59.474325] nix-daemon[856]: reaped child process 1066, status = succeeded4589lb # [ 59.482961] niks3-server[937]: 2026/09/22 15:46:12 INFO Received uploads request method=POST path=/api/pending_closures4590worker2 # [ 59.497297] nix-grpc-daemon[910]: time=2026-09-22T15:46:12.096Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4591worker2 # [ 59.498782] nix-grpc-daemon[910]: time=2026-09-22T15:46:12.096Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4592lb # [ 59.511842] niks3-server[937]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4593lb # [ 59.521166] niks3-server[937]: 2026/09/22 15:46:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4594worker2 # [ 59.525414] nix-grpc-daemon[910]: time=2026-09-22T15:46:12.124Z level=INFO msg="Uploading 1 narinfos"4595lb # [ 59.525219] niks3-server[937]: 2026/09/22 15:46:12 INFO Signed narinfos id=17 count=14596lb # [ 59.532370] niks3-server[937]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4597lb # [ 59.540000] niks3-server[937]: 2026/09/22 15:46:12 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4598worker2 # [ 59.543898] nix-grpc-daemon[910]: time=2026-09-22T15:46:12.143Z level=INFO msg="Upload complete. (351ms)"4599lb # [ 59.543840] niks3-server[937]: 2026/09/22 15:46:12 INFO Completed upload id=174600lb # [ 59.546989] niks3-server[937]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4601worker2 # [ 59.552512] nix-grpc-daemon[772]: ts=2026-09-22T15:46:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=4244602worker2 # [ 59.560618] nix-grpc-daemon[772]: ts=2026-09-22T15:46:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484603worker2 # [ 59.571433] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4604worker2 # [ 59.573516] nix-daemon[856]: reaped child process 1056, status = succeeded4605worker2 # [ 59.584639] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4606worker2 # [ 65.599864] nix-grpc-daemon[772]: ts=2026-09-22T15:46:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04607worker2 # [ 65.633220] nix-daemon[856]: reaped child process 1068, status = succeeded4608worker2 # [ 65.637855] nix-daemon[856]: reaped child process 1070, status = succeeded4609worker1 # [ 65.904887] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:416764610worker2 # [ 65.896164] nix-grpc-daemon[772]: ts=2026-09-22T15:46:18Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04611worker1 # [ 65.927927] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764612worker2 # [ 65.902313] nix-grpc-daemon[772]: ts=2026-09-22T15:46:18Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14613worker1 # [ 65.943437] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416764614worker1 # [ 65.957507] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416764615worker1 # [ 65.973779] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4616worker1 # [ 66.132867] nix-daemon[829]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4617worker1 # [ 66.155655] nix-daemon[829]: reaped child process 1075, status = succeeded4618worker1 # [ 66.256507] nix-daemon[829]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4619worker1 # [ 66.278486] nix-daemon[829]: reaped child process 1080, status = succeeded4620lb # [ 66.250915] niks3-server[937]: 2026/09/22 15:46:18 INFO Received uploads request method=POST path=/api/pending_closures4621worker1 # [ 66.287260] nix-grpc-daemon[842]: time=2026-09-22T15:46:18.889Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4622worker1 # [ 66.288934] nix-grpc-daemon[842]: time=2026-09-22T15:46:18.889Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4623lb # [ 66.285918] niks3-server[937]: 2026/09/22 15:46:18 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4624lb # [ 66.303737] niks3-server[937]: 2026/09/22 15:46:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4625worker1 # [ 66.335667] nix-grpc-daemon[842]: time=2026-09-22T15:46:18.938Z level=INFO msg="Uploading 1 narinfos"4626lb # [ 66.313994] niks3-server[937]: 2026/09/22 15:46:18 INFO Signed narinfos id=18 count=14627lb # [ 66.319996] niks3-server[937]: 2026/09/22 15:46:18 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4628lb # [ 66.331783] niks3-server[937]: 2026/09/22 15:46:18 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4629worker1 # [ 66.363008] nix-grpc-daemon[842]: time=2026-09-22T15:46:18.965Z level=INFO msg="Upload complete. (374ms)"4630lb # [ 66.339380] niks3-server[937]: 2026/09/22 15:46:18 INFO Completed upload id=184631lb # [ 66.344478] niks3-server[937]: 2026/09/22 15:46:18 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4632worker1 # [ 66.372428] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=4244633worker1 # [ 66.380433] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764634worker1 # [ 66.394472] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4635worker1 # [ 66.397676] nix-daemon[829]: reaped child process 1070, status = succeeded4636worker1 # [ 66.410077] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4637lb # [ 68.538597] envoy[894]: conn peer=[2001:db8:1::1]:54778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=954 ms=65624638worker1 # [ 72.425383] nix-grpc-daemon[765]: ts=2026-09-22T15:46:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04639worker1 # [ 72.464798] nix-daemon[829]: reaped child process 1082, status = succeeded4640worker1 # [ 72.467545] nix-daemon[829]: reaped child process 1084, status = succeeded4641(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.76 seconds)4642subtest: the first client leaving does not fail the second4643client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4644client # [ 72.695953] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4645client # Running as unit: share1.service; invocation ID: 25a5d8b4297648e2be3083ba6a631e5b4646client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4647client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4648client # [ 72.897533] nix[1021]: don't know how to build these paths:4649client # [ 72.898494] nix[1021]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4650worker1 # [ 72.915196] nix-grpc-daemon[765]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:416764651worker2 # [ 72.905214] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04652worker2 # [ 72.913255] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14653worker2 # [ 72.921159] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484654worker2 # [ 72.932793] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484655client # [ 72.950500] nix[1021]: copying 1 paths to 'lb:50051'...4656client # [ 72.953213] nix[1021]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4657worker2 # [ 72.947842] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:603484658worker2 # [ 72.966237] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4659worker2 # [ 73.118249] nix-daemon[856]: accepted connection from pid 1097, user nix-grpc-daemon (trusted)4660worker2 # [ 73.138464] nix-daemon[856]: reaped child process 1100, status = succeeded4661worker2 # [ 73.242298] nix-daemon[856]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)4662worker2 # [ 73.262265] nix-daemon[856]: reaped child process 1105, status = succeeded4663lb # [ 73.260788] niks3-server[937]: 2026/09/22 15:46:25 INFO Received uploads request method=POST path=/api/pending_closures4664worker2 # [ 73.271707] nix-grpc-daemon[910]: time=2026-09-22T15:46:25.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4665worker2 # [ 73.273401] nix-grpc-daemon[910]: time=2026-09-22T15:46:25.870Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4666lb # [ 73.292788] niks3-server[937]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4667lb # [ 73.309894] niks3-server[937]: 2026/09/22 15:46:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4668worker2 # [ 73.316319] nix-grpc-daemon[910]: time=2026-09-22T15:46:25.915Z level=INFO msg="Uploading 1 narinfos"4669lb # [ 73.317951] niks3-server[937]: 2026/09/22 15:46:25 INFO Signed narinfos id=19 count=14670lb # [ 73.325345] niks3-server[937]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4671worker2 # [ 73.346169] nix-grpc-daemon[910]: time=2026-09-22T15:46:25.945Z level=INFO msg="Upload complete. (365ms)"4672lb # [ 73.340729] niks3-server[937]: 2026/09/22 15:46:25 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4673lb # [ 73.351653] niks3-server[937]: 2026/09/22 15:46:25 INFO Completed upload id=194674worker2 # [ 73.356316] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=4324675lb # [ 73.356515] niks3-server[937]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4676worker2 # [ 73.368905] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484677worker2 # [ 73.377443] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4678worker2 # [ 73.383208] nix-daemon[856]: reaped child process 1095, status = succeeded4679worker2 # [ 73.394588] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4680client # [ 73.399728] nix[1021]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4681client # [ 73.801581] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4682client # Running as unit: share2.service; invocation ID: 4dfb4e9b1331469d864c6c3806bf10c64683client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4684worker1 # [ 74.037205] nix-grpc-daemon[765]: ts=2026-09-22T15:46:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:416764685client # [ 74.025692] nix[1037]: don't know how to build these paths:4686client # [ 74.027619] nix[1037]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4687worker2 # [ 74.033301] nix-grpc-daemon[772]: ts=2026-09-22T15:46:26Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04688client # [ 74.048324] nix[1037]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4689worker2 # [ 74.041029] nix-grpc-daemon[772]: ts=2026-09-22T15:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484690client: must succeed: sleep 2; systemctl kill -s INT share14691client # [ 77.181319] systemd[1]: share1.service: Sent signal SIGINT to main process 1021 (nix) on client request.4692client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4693client # [ 77.190721] nix[1021]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4694client: waiting for success: ! systemctl is-active share14695client # [ 77.205130] nix[1021]: error: interrupted by the user4696client # [ 77.268326] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4697client # [ 77.271866] systemd[1]: share1.service: Failed with result 'exit-code'.4698client # [ 77.274168] systemd[1]: share1.service: Consumed 173ms CPU time over 4.575s wall clock time, 39.4M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic.4699client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4700lb # [ 78.537824] envoy[894]: conn peer=[2001:db8:1::1]:35158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=951 ms=65834701lb # [ 78.542803] envoy[894]: conn peer=[2001:db8:1::1]:35650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2990 tx=789 ms=43144702client: waiting for success: ! systemctl is-active share24703worker2 # [ 78.624514] nix-daemon[856]: reaped child process 1109, status = succeeded4704worker2 # [ 78.724210] nix-daemon[856]: accepted connection from pid 1145, user nix-grpc-daemon (trusted)4705worker2 # [ 78.742314] nix-daemon[856]: reaped child process 1148, status = succeeded4706lb # [ 78.748973] niks3-server[937]: 2026/09/22 15:46:31 INFO Received uploads request method=POST path=/api/pending_closures4707worker2 # [ 78.760145] nix-grpc-daemon[910]: time=2026-09-22T15:46:31.358Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4708worker2 # [ 78.761639] nix-grpc-daemon[910]: time=2026-09-22T15:46:31.358Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4709lb # [ 78.790810] niks3-server[937]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4710lb # [ 78.804176] niks3-server[937]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4711worker2 # [ 78.818054] nix-grpc-daemon[910]: time=2026-09-22T15:46:31.417Z level=INFO msg="Uploading 1 narinfos"4712lb # [ 78.819625] niks3-server[937]: 2026/09/22 15:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4713lb # [ 78.828861] niks3-server[937]: 2026/09/22 15:46:31 INFO Signed narinfos id=20 count=14714lb # [ 78.831330] niks3-server[937]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4715lb # [ 78.845365] niks3-server[937]: 2026/09/22 15:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4716worker2 # [ 78.853739] nix-grpc-daemon[910]: time=2026-09-22T15:46:31.453Z level=INFO msg="Upload complete. (241ms)"4717lb # [ 78.854111] niks3-server[937]: 2026/09/22 15:46:31 INFO Completed upload id=204718lb # [ 78.860752] niks3-server[937]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4719worker2 # [ 78.867837] nix-grpc-daemon[772]: ts=2026-09-22T15:46:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14720worker2 # [ 78.884397] nix-grpc-daemon[772]: ts=2026-09-22T15:46:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14721worker2 # [ 78.893076] nix-daemon[856]: reaped child process 1107, status = succeeded4722client # [ 78.909881] systemd[1]: share2.service: Deactivated successfully.4723client # [ 78.915001] systemd[1]: share2.service: Consumed 190ms CPU time over 5.109s wall clock time, 33.3M memory peak, 6.2K incoming IP traffic, 5.8K outgoing IP traffic.4724client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4725client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4726client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4727(finished: subtest: the first client leaving does not fail the second, in 7.11 seconds)4728subtest: build hook: nix-daemon with builders = grpc://lb4729client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4730client # [ 79.874037] systemd[1]: Started Nix Daemon.4731client # [ 79.995800] nix-daemon[1089]: accepted connection from pid 1086, user root (trusted)4732worker2 # [ 80.142429] nix-grpc-daemon[772]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:603484733worker1 # [ 80.173423] nix-grpc-daemon[765]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416764734worker2 # [ 80.159273] nix-grpc-daemon[772]: ts=2026-09-22T15:46:32Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04735worker2 # [ 80.165903] nix-grpc-daemon[772]: ts=2026-09-22T15:46:32Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14736worker1 # [ 80.194628] nix-grpc-daemon[765]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764737worker1 # [ 80.209875] nix-grpc-daemon[765]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416764738worker1 # [ 80.224608] nix-grpc-daemon[765]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416764739worker1 # [ 80.239712] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4740worker1 # [ 80.400949] nix-daemon[829]: accepted connection from pid 1123, user nix-grpc-daemon (trusted)4741worker1 # [ 80.422330] nix-daemon[829]: reaped child process 1126, status = succeeded4742worker1 # [ 80.518135] nix-daemon[829]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)4743worker1 # [ 80.535660] nix-daemon[829]: reaped child process 1131, status = succeeded4744lb # [ 80.513805] niks3-server[937]: 2026/09/22 15:46:33 INFO Received uploads request method=POST path=/api/pending_closures4745worker1 # [ 80.549656] nix-grpc-daemon[842]: time=2026-09-22T15:46:33.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4746worker1 # [ 80.551127] nix-grpc-daemon[842]: time=2026-09-22T15:46:33.152Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4747lb # [ 80.539681] niks3-server[937]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4748lb # [ 80.553775] niks3-server[937]: 2026/09/22 15:46:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4749worker1 # [ 80.586370] nix-grpc-daemon[842]: time=2026-09-22T15:46:33.189Z level=INFO msg="Uploading 1 narinfos"4750lb # [ 80.563877] niks3-server[937]: 2026/09/22 15:46:33 INFO Signed narinfos id=21 count=14751lb # [ 80.573867] niks3-server[937]: 2026/09/22 15:46:33 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4752lb # [ 80.586706] niks3-server[937]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4753lb # [ 80.595799] niks3-server[937]: 2026/09/22 15:46:33 INFO Completed upload id=214754worker1 # [ 80.622369] nix-grpc-daemon[842]: time=2026-09-22T15:46:33.225Z level=INFO msg="Upload complete. (365ms)"4755lb # [ 80.600184] niks3-server[937]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4756worker1 # [ 80.632256] nix-grpc-daemon[765]: ts=2026-09-22T15:46:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=5124757worker1 # [ 80.647956] nix-grpc-daemon[765]: ts=2026-09-22T15:46:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764758worker1 # [ 80.654536] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4759worker1 # [ 80.659263] nix-daemon[829]: reaped child process 1121, status = succeeded4760worker1 # [ 80.672137] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4761worker1 # [ 82.135389] nix-daemon[829]: reaped child process 1135, status = succeeded4762worker1 # [ 82.251874] nix-daemon[829]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)4763worker1 # [ 82.271730] nix-daemon[829]: reaped child process 1154, status = succeeded4764lb # [ 82.249892] niks3-server[937]: 2026/09/22 15:46:34 INFO Received uploads request method=POST path=/api/pending_closures4765worker1 # [ 82.287789] nix-grpc-daemon[842]: time=2026-09-22T15:46:34.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4766worker1 # [ 82.289556] nix-grpc-daemon[842]: time=2026-09-22T15:46:34.888Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4767lb # [ 82.287482] niks3-server[937]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4768lb # [ 82.301781] niks3-server[937]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4769lb # [ 82.314243] niks3-server[937]: 2026/09/22 15:46:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4770worker1 # [ 82.348051] nix-grpc-daemon[842]: time=2026-09-22T15:46:34.950Z level=INFO msg="Uploading 1 narinfos"4771lb # [ 82.327091] niks3-server[937]: 2026/09/22 15:46:34 INFO Signed narinfos id=22 count=14772lb # [ 82.333149] niks3-server[937]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4773lb # [ 82.342940] niks3-server[937]: 2026/09/22 15:46:34 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4774lb # [ 82.351884] niks3-server[937]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4775worker1 # [ 82.384317] nix-grpc-daemon[842]: time=2026-09-22T15:46:34.987Z level=INFO msg="Upload complete. (302ms)"4776lb # [ 82.359568] niks3-server[937]: 2026/09/22 15:46:34 INFO Completed upload id=224777worker1 # [ 82.397264] nix-grpc-daemon[765]: ts=2026-09-22T15:46:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14778worker1 # [ 82.418005] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:416764779worker1 # [ 82.423561] nix-daemon[829]: reaped child process 1133, status = succeeded4780worker1 # [ 82.439471] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:416764781worker1 # [ 82.446305] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=1284782worker1 # [ 82.723281] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:416764783worker2 # [ 82.703233] nix-grpc-daemon[772]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484784worker2 # [ 82.729940] nix-grpc-daemon[772]: ts=2026-09-22T15:46:35Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04785worker2 # [ 82.736338] nix-grpc-daemon[772]: ts=2026-09-22T15:46:35Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14786worker1 # [ 82.765013] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764787worker1 # [ 82.779984] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416764788worker1 # [ 82.795165] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.397Z level=INFO msg="All 1 paths already cached"4789worker1 # [ 82.805073] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416764790worker1 # [ 82.819751] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4791worker1 # [ 82.980129] nix-daemon[829]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4792worker1 # [ 83.007786] nix-daemon[829]: reaped child process 1161, status = succeeded4793worker1 # [ 83.098028] nix-daemon[829]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4794worker1 # [ 83.115885] nix-daemon[829]: reaped child process 1166, status = succeeded4795lb # [ 83.094426] niks3-server[937]: 2026/09/22 15:46:35 INFO Received uploads request method=POST path=/api/pending_closures4796worker1 # [ 83.130538] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.733Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4797worker1 # [ 83.132067] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.733Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4798lb # [ 83.122522] niks3-server[937]: 2026/09/22 15:46:35 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4799worker1 # [ 83.170749] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.773Z level=INFO msg="Uploading 1 narinfos"4800lb # [ 83.137725] niks3-server[937]: 2026/09/22 15:46:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4801lb # [ 83.150915] niks3-server[937]: 2026/09/22 15:46:35 INFO Signed narinfos id=23 count=14802lb # [ 83.159238] niks3-server[937]: 2026/09/22 15:46:35 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4803worker1 # [ 83.198577] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.801Z level=INFO msg="Upload complete. (361ms)"4804lb # [ 83.172737] niks3-server[937]: 2026/09/22 15:46:35 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4805lb # [ 83.181478] niks3-server[937]: 2026/09/22 15:46:35 INFO Completed upload id=234806worker1 # [ 83.208467] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=6244807lb # [ 83.185454] niks3-server[937]: 2026/09/22 15:46:35 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4808worker1 # [ 83.219146] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764809worker1 # [ 83.228500] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4810worker1 # [ 83.230706] nix-daemon[829]: reaped child process 1156, status = succeeded4811worker1 # [ 83.242504] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4812worker1 # [ 84.736805] nix-daemon[829]: reaped child process 1170, status = succeeded4813worker1 # [ 84.803069] nix-daemon[829]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)4814worker1 # [ 84.820912] nix-daemon[829]: reaped child process 1189, status = succeeded4815lb # [ 84.801669] niks3-server[937]: 2026/09/22 15:46:37 INFO Received uploads request method=POST path=/api/pending_closures4816worker1 # [ 84.836550] nix-grpc-daemon[842]: time=2026-09-22T15:46:37.439Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4817worker1 # [ 84.840280] nix-grpc-daemon[842]: time=2026-09-22T15:46:37.441Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4818lb # [ 84.839120] niks3-server[937]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4819lb # [ 84.852665] niks3-server[937]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4820worker1 # [ 84.894567] nix-grpc-daemon[842]: time=2026-09-22T15:46:37.497Z level=INFO msg="Uploading 1 narinfos"4821lb # [ 84.869076] niks3-server[937]: 2026/09/22 15:46:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4822lb # [ 84.880678] niks3-server[937]: 2026/09/22 15:46:37 INFO Signed narinfos id=24 count=14823lb # [ 84.887423] niks3-server[937]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4824lb # [ 84.897074] niks3-server[937]: 2026/09/22 15:46:37 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4825lb # [ 84.905333] niks3-server[937]: 2026/09/22 15:46:37 INFO Completed upload id=244826worker1 # [ 84.931440] nix-grpc-daemon[842]: time=2026-09-22T15:46:37.534Z level=INFO msg="Upload complete. (276ms)"4827lb # [ 84.909062] niks3-server[937]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4828worker1 # [ 84.945117] nix-grpc-daemon[765]: ts=2026-09-22T15:46:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14829worker1 # [ 84.964361] nix-grpc-daemon[765]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:416764830worker1 # [ 84.970588] nix-daemon[829]: reaped child process 1168, status = succeeded4831worker2 # [ 84.970170] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:603484832worker2 # [ 84.985347] nix-grpc-daemon[772]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4833worker2 # [ 85.003373] nix-grpc-daemon[772]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4834worker2 # [ 85.018890] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_out=1764835worker2 # [ 85.304248] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:603484836worker2 # [ 85.311682] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484837worker2 # [ 85.319297] nix-grpc-daemon[910]: time=2026-09-22T15:46:37.916Z level=INFO msg="All 2 paths already cached"4838worker2 # [ 85.330778] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04839worker2 # [ 85.337290] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14840worker1 # [ 85.369699] nix-grpc-daemon[765]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764841worker1 # [ 85.385371] nix-grpc-daemon[765]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416764842worker1 # [ 85.399547] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.002Z level=INFO msg="All 2 paths already cached"4843worker1 # [ 85.410096] nix-grpc-daemon[765]: ts=2026-09-22T15:46:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416764844worker1 # [ 85.424112] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4845worker1 # [ 85.581765] nix-daemon[829]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)4846worker1 # [ 85.598526] nix-daemon[829]: reaped child process 1196, status = succeeded4847worker1 # [ 85.701274] nix-daemon[829]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)4848worker1 # [ 85.719757] nix-daemon[829]: reaped child process 1201, status = succeeded4849lb # [ 85.692004] niks3-server[937]: 2026/09/22 15:46:38 INFO Received uploads request method=POST path=/api/pending_closures4850worker1 # [ 85.728223] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.330Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4851worker1 # [ 85.729660] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.330Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4852lb # [ 85.722819] niks3-server[937]: 2026/09/22 15:46:38 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4853lb # [ 85.738100] niks3-server[937]: 2026/09/22 15:46:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4854worker1 # [ 85.771388] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.374Z level=INFO msg="Uploading 1 narinfos"4855lb # [ 85.750927] niks3-server[937]: 2026/09/22 15:46:38 INFO Signed narinfos id=25 count=14856lb # [ 85.756578] niks3-server[937]: 2026/09/22 15:46:38 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4857lb # [ 85.768277] niks3-server[937]: 2026/09/22 15:46:38 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4858lb # [ 85.775036] niks3-server[937]: 2026/09/22 15:46:38 INFO Completed upload id=254859worker1 # [ 85.801581] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.404Z level=INFO msg="Upload complete. (360ms)"4860lb # [ 85.778393] niks3-server[937]: 2026/09/22 15:46:38 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4861worker1 # [ 85.808184] nix-grpc-daemon[765]: ts=2026-09-22T15:46:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=7524862worker1 # [ 85.820406] nix-grpc-daemon[765]: ts=2026-09-22T15:46:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416764863worker1 # [ 85.828896] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4864worker1 # [ 85.833817] nix-daemon[829]: reaped child process 1191, status = succeeded4865worker1 # [ 85.844241] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4866worker1 # [ 86.501936] nix-daemon[829]: reaped child process 1205, status = succeeded4867worker1 # [ 86.622751] nix-daemon[829]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)4868worker1 # [ 86.641281] nix-daemon[829]: reaped child process 1224, status = succeeded4869lb # [ 86.621556] niks3-server[937]: 2026/09/22 15:46:39 INFO Received uploads request method=POST path=/api/pending_closures4870worker1 # [ 86.659212] nix-grpc-daemon[842]: time=2026-09-22T15:46:39.260Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4871worker1 # [ 86.661060] nix-grpc-daemon[842]: time=2026-09-22T15:46:39.260Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4872lb # [ 86.653962] niks3-server[937]: 2026/09/22 15:46:39 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4873lb # [ 86.674388] niks3-server[937]: 2026/09/22 15:46:39 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4874worker1 # [ 86.710841] nix-grpc-daemon[842]: time=2026-09-22T15:46:39.313Z level=INFO msg="Uploading 1 narinfos"4875lb # [ 86.685924] niks3-server[937]: 2026/09/22 15:46:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4876lb # [ 86.693340] niks3-server[937]: 2026/09/22 15:46:39 INFO Signed narinfos id=26 count=14877lb # [ 86.699723] niks3-server[937]: 2026/09/22 15:46:39 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4878lb # [ 86.711758] niks3-server[937]: 2026/09/22 15:46:39 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4879lb # [ 86.718202] niks3-server[937]: 2026/09/22 15:46:39 INFO Completed upload id=264880worker1 # [ 86.743143] nix-grpc-daemon[842]: time=2026-09-22T15:46:39.345Z level=INFO msg="Upload complete. (287ms)"4881lb # [ 86.721563] niks3-server[937]: 2026/09/22 15:46:39 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4882worker1 # [ 86.757278] nix-grpc-daemon[765]: ts=2026-09-22T15:46:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14883worker1 # [ 86.779704] nix-grpc-daemon[765]: ts=2026-09-22T15:46:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:416764884worker1 # [ 86.784665] nix-daemon[829]: reaped child process 1203, status = succeeded4885worker1 # [ 86.793503] nix-grpc-daemon[765]: ts=2026-09-22T15:46:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:416764886worker1 # [ 86.800073] nix-grpc-daemon[765]: ts=2026-09-22T15:46:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=2324887client # [ 86.919795] nix-daemon[1089]: reaped child process 1091, status = succeeded4888client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.20 seconds)4889client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4890client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4891client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4892client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4893client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4894client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4895(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.32 seconds)4896subtest: an input only one worker has still reaches the builder4897client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4898client # this derivation will be built:4899client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4900client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4901client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.35 seconds)4902client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4903client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.17 seconds)4904worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4905worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.30 seconds)4906worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4907worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.06 seconds)4908client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24909client # [ 88.255617] nix-daemon[1089]: accepted connection from pid 1172, user root (trusted)4910client # this derivation will be built:4911client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4912worker1 # [ 88.500230] nix-grpc-daemon[765]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:416764913client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4914worker2 # [ 88.482709] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484915client # copying 1 paths...4916client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4917worker1 # [ 88.538960] nix-grpc-daemon[765]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416764918worker1 # [ 88.563600] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4919lb # [ 88.539127] envoy[894]: conn peer=[2001:db8:1::1]:35652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=48694920lb # [ 88.544839] envoy[894]: conn peer=[2001:db8:1::1]:35666 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3417 tx=1942 ms=23364921lb # [ 88.550063] envoy[894]: conn peer=[2001:db8:1::1]:51814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=744922lb # [ 88.554865] envoy[894]: conn peer=[2001:db8:1::1]:51824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2156 ms=23794923lb # [ 88.559875] envoy[894]: conn peer=[2001:db8:1::1]:51828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=1144924lb # [ 88.565897] envoy[894]: conn peer=[2001:db8:1::1]:51842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4296 tx=2397 ms=15314925lb # [ 88.577111] envoy[894]: conn peer=[2001:db8:1::1]:51846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=724926worker1 # [ 88.871908] nix-daemon[829]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)4927worker1 # [ 88.906213] nix-daemon[829]: reaped child process 1236, status = succeeded4928lb # [ 88.891718] niks3-server[937]: 2026/09/22 15:46:41 INFO Received uploads request method=POST path=/api/pending_closures4929worker1 # [ 88.934427] nix-grpc-daemon[842]: time=2026-09-22T15:46:41.535Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4930worker1 # [ 88.936421] nix-grpc-daemon[842]: time=2026-09-22T15:46:41.535Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4931lb # [ 88.953885] niks3-server[937]: 2026/09/22 15:46:41 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4932lb # [ 88.971501] niks3-server[937]: 2026/09/22 15:46:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4933worker1 # [ 89.007074] nix-grpc-daemon[842]: time=2026-09-22T15:46:41.608Z level=INFO msg="Uploading 1 narinfos"4934lb # [ 88.984618] niks3-server[937]: 2026/09/22 15:46:41 INFO Signed narinfos id=27 count=14935lb # [ 88.991196] niks3-server[937]: 2026/09/22 15:46:41 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4936lb # [ 89.011368] niks3-server[937]: 2026/09/22 15:46:41 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4937lb # [ 89.020075] niks3-server[937]: 2026/09/22 15:46:41 INFO Completed upload id=274938worker1 # [ 89.044965] nix-grpc-daemon[842]: time=2026-09-22T15:46:41.647Z level=INFO msg="Upload complete. (453ms)"4939lb # [ 89.023153] niks3-server[937]: 2026/09/22 15:46:41 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4940worker1 # [ 89.056548] nix-grpc-daemon[765]: ts=2026-09-22T15:46:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=1204941worker1 # [ 89.088304] nix-daemon[829]: reaped child process 1231, status = succeeded4942worker2 # [ 89.062278] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04943worker2 # [ 89.072993] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14944worker2 # [ 89.112462] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484945worker2 # [ 89.137259] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603484946client # copying 2 paths to 'lb:50051'...4947worker2 # [ 89.168008] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:603484948client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4949client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4950worker2 # [ 89.196943] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4951worker2 # [ 89.361675] nix-daemon[856]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)4952worker2 # [ 89.385878] nix-daemon[856]: reaped child process 1168, status = succeeded4953worker2 # [ 89.485460] nix-daemon[856]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)4954worker2 # [ 89.503988] nix-daemon[856]: reaped child process 1173, status = succeeded4955lb # [ 89.502574] niks3-server[937]: 2026/09/22 15:46:42 INFO Received uploads request method=POST path=/api/pending_closures4956lb # [ 89.512448] niks3-server[937]: 2026/09/22 15:46:42 INFO Received uploads request method=POST path=/api/pending_closures4957worker2 # [ 89.524394] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.123Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4958worker2 # [ 89.525923] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.123Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4959worker2 # [ 89.527694] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.123Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4960lb # [ 89.556126] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4961lb # [ 89.569787] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4962lb # [ 89.581970] niks3-server[937]: 2026/09/22 15:46:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4963worker2 # [ 89.588647] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.187Z level=INFO msg="Uploading 3 narinfos"4964lb # [ 89.590623] niks3-server[937]: 2026/09/22 15:46:42 INFO Signed narinfos id=28 count=14965lb # [ 89.596557] niks3-server[937]: 2026/09/22 15:46:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4966lb # [ 89.599545] niks3-server[937]: 2026/09/22 15:46:42 INFO Signed narinfos id=29 count=24967lb # [ 89.606595] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4968lb # [ 89.613596] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4969lb # [ 89.625747] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4970lb # [ 89.635298] niks3-server[937]: 2026/09/22 15:46:42 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4971lb # [ 89.641337] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4972lb # [ 89.644459] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4973lb # [ 89.648273] niks3-server[937]: 2026/09/22 15:46:42 INFO Completed upload id=284974worker2 # [ 89.649152] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.248Z level=INFO msg="Upload complete. (431ms)"4975lb # [ 89.650435] niks3-server[937]: 2026/09/22 15:46:42 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4976lb # [ 89.653811] niks3-server[937]: 2026/09/22 15:46:42 INFO Completed upload id=294977worker2 # [ 89.661437] nix-grpc-daemon[772]: ts=2026-09-22T15:46:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=2 nar_bytes_in=9844978worker2 # [ 89.676704] nix-grpc-daemon[772]: ts=2026-09-22T15:46:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603484979worker2 # [ 89.691021] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4980worker2 # [ 89.694992] nix-daemon[856]: reaped child process 1163, status = succeeded4981worker2 # [ 89.709603] nix-grpc-daemon[772]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4982worker2 # [ 89.722803] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)4983client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4984client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4985client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4986worker2 # [ 91.230899] nix-daemon[856]: reaped child process 1178, status = succeeded4987worker2 # [ 91.302686] nix-daemon[856]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)4988worker2 # [ 91.320220] nix-daemon[856]: reaped child process 1197, status = succeeded4989lb # [ 91.324710] niks3-server[937]: 2026/09/22 15:46:43 INFO Received uploads request method=POST path=/api/pending_closures4990worker2 # [ 91.335979] nix-grpc-daemon[910]: time=2026-09-22T15:46:43.933Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4991lb # [ 91.356072] niks3-server[937]: 2026/09/22 15:46:43 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4992worker2 # [ 91.368298] nix-grpc-daemon[910]: time=2026-09-22T15:46:43.967Z level=INFO msg="Uploading 1 narinfos"4993lb # [ 91.373219] niks3-server[937]: 2026/09/22 15:46:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign4994lb # [ 91.382192] niks3-server[937]: 2026/09/22 15:46:43 INFO Signed narinfos id=30 count=14995lb # [ 91.389722] niks3-server[937]: 2026/09/22 15:46:43 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete4996lb # [ 91.397988] niks3-server[937]: 2026/09/22 15:46:43 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4997lb # [ 91.404715] niks3-server[937]: 2026/09/22 15:46:43 INFO Completed upload id=304998worker2 # [ 91.405765] nix-grpc-daemon[910]: time=2026-09-22T15:46:44.005Z level=INFO msg="Upload complete. (269ms)"4999lb # [ 91.408915] niks3-server[937]: 2026/09/22 15:46:43 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5000worker2 # [ 91.417059] nix-grpc-daemon[772]: ts=2026-09-22T15:46:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15001worker2 # [ 91.435595] nix-daemon[856]: reaped child process 1175, status = succeeded5002client # copying 1 paths...5003worker1 # [ 91.489879] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:416765004worker1 # [ 91.503842] nix-grpc-daemon[765]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...5005client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5006worker1 # [ 91.550431] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:416765007worker1 # [ 91.557633] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=1205008client # [ 91.665336] nix-daemon[1089]: reaped child process 1175, status = succeeded5009client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.74 seconds)5010client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25011client # [ 91.833400] nix-daemon[1089]: accepted connection from pid 1193, user root (trusted)5012client # this derivation will be built:5013client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5014worker1 # [ 91.997665] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:416765015client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5016worker1 # [ 92.006025] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416765017worker1 # [ 92.014670] nix-grpc-daemon[842]: time=2026-09-22T15:46:44.614Z level=INFO msg="All 1 paths already cached"5018worker2 # [ 91.999988] nix-grpc-daemon[772]: ts=2026-09-22T15:46:44Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05019worker2 # [ 92.007398] nix-grpc-daemon[772]: ts=2026-09-22T15:46:44Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15020worker1 # [ 92.070887] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416765021worker1 # [ 92.087165] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416765022client # copying 1 paths to 'lb:50051'...5023client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5024worker1 # [ 92.108659] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416765025worker1 # [ 92.124774] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5026worker1 # [ 92.148734] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5)5027worker1 # [ 92.254518] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 2/5)5028worker1 # [ 92.322528] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 3/5)5029worker1 # [ 92.459506] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 467 ms (attempt 4/5)5030worker1 # [ 92.930058] nix-grpc-daemon[765]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5031worker1 # [ 92.938706] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5032worker1 # [ 92.946294] nix-grpc-daemon[765]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5033worker1 # [ 93.122337] nix-daemon[829]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5034worker1 # [ 93.141542] nix-daemon[829]: reaped child process 1249, status = succeeded5035worker1 # [ 93.247511] nix-daemon[829]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5036worker1 # [ 93.262227] nix-daemon[829]: reaped child process 1254, status = succeeded5037lb # [ 93.244812] niks3-server[937]: 2026/09/22 15:46:45 INFO Received uploads request method=POST path=/api/pending_closures5038worker1 # [ 93.280991] nix-grpc-daemon[842]: time=2026-09-22T15:46:45.883Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5039worker1 # [ 93.282436] nix-grpc-daemon[842]: time=2026-09-22T15:46:45.883Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5040lb # [ 93.273199] niks3-server[937]: 2026/09/22 15:46:45 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5041worker1 # [ 93.312712] nix-grpc-daemon[842]: time=2026-09-22T15:46:45.915Z level=INFO msg="Uploading 1 narinfos"5042lb # [ 93.287254] niks3-server[937]: 2026/09/22 15:46:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5043lb # [ 93.299163] niks3-server[937]: 2026/09/22 15:46:45 INFO Signed narinfos id=31 count=15044lb # [ 93.304973] niks3-server[937]: 2026/09/22 15:46:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5045worker1 # [ 93.339254] nix-grpc-daemon[842]: time=2026-09-22T15:46:45.942Z level=INFO msg="Upload complete. (379ms)"5046lb # [ 93.314757] niks3-server[937]: 2026/09/22 15:46:45 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5047lb # [ 93.321232] niks3-server[937]: 2026/09/22 15:46:45 INFO Completed upload id=315048worker1 # [ 93.348255] nix-grpc-daemon[765]: ts=2026-09-22T15:46:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 paths=1 nar_bytes_in=5685049lb # [ 93.323598] niks3-server[937]: 2026/09/22 15:46:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5050worker1 # [ 93.355158] nix-grpc-daemon[765]: ts=2026-09-22T15:46:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416765051worker1 # [ 93.366213] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5052worker1 # [ 93.370750] nix-daemon[829]: reaped child process 1239, status = succeeded5053client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5054worker1 # [ 93.382404] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted)5055client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5056client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5057worker1 # [ 94.689913] nix-daemon[829]: reaped child process 1258, status = succeeded5058worker1 # [ 94.759624] nix-daemon[829]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5059worker1 # [ 94.775855] nix-daemon[829]: reaped child process 1277, status = succeeded5060lb # [ 94.757578] niks3-server[937]: 2026/09/22 15:46:47 INFO Received uploads request method=POST path=/api/pending_closures5061worker1 # [ 94.793145] nix-grpc-daemon[842]: time=2026-09-22T15:46:47.395Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5062lb # [ 94.789861] niks3-server[937]: 2026/09/22 15:46:47 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5063worker1 # [ 94.827514] nix-grpc-daemon[842]: time=2026-09-22T15:46:47.430Z level=INFO msg="Uploading 1 narinfos"5064lb # [ 94.802746] niks3-server[937]: 2026/09/22 15:46:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5065lb # [ 94.814285] niks3-server[937]: 2026/09/22 15:46:47 INFO Signed narinfos id=32 count=15066lb # [ 94.819229] niks3-server[937]: 2026/09/22 15:46:47 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5067lb # [ 94.831769] niks3-server[937]: 2026/09/22 15:46:47 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5068worker1 # [ 94.856986] nix-grpc-daemon[842]: time=2026-09-22T15:46:47.459Z level=INFO msg="Upload complete. (264ms)"5069lb # [ 94.839749] niks3-server[937]: 2026/09/22 15:46:47 INFO Completed upload id=325070lb # [ 94.843683] niks3-server[937]: 2026/09/22 15:46:47 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5071worker1 # [ 94.869202] nix-grpc-daemon[765]: ts=2026-09-22T15:46:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15072worker1 # [ 94.884705] nix-daemon[829]: reaped child process 1256, status = succeeded5073client # copying 1 paths...5074worker2 # [ 94.893376] nix-grpc-daemon[772]: ts=2026-09-22T15:46:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:603485075worker2 # [ 94.905796] nix-grpc-daemon[772]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5076client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5077worker1 # [ 94.975421] nix-grpc-daemon[765]: ts=2026-09-22T15:46:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:416765078worker1 # [ 94.983840] nix-grpc-daemon[765]: ts=2026-09-22T15:46:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=1205079client # [ 95.052681] nix-daemon[1089]: reaped child process 1196, status = succeeded5080client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.39 seconds)5081client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25082client # [ 95.229522] nix-daemon[1089]: accepted connection from pid 1215, user root (trusted)5083client # this derivation will be built:5084client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5085worker2 # [ 95.368130] nix-grpc-daemon[772]: ts=2026-09-22T15:46:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:603485086client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5087worker1 # [ 95.403046] nix-grpc-daemon[765]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416765088worker1 # [ 95.412817] nix-grpc-daemon[842]: time=2026-09-22T15:46:48.015Z level=INFO msg="All 1 paths already cached"5089worker2 # [ 95.405010] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05090worker2 # [ 95.412896] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15091worker2 # [ 95.422773] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603485092worker2 # [ 95.434054] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603485093client # copying 1 paths to 'lb:50051'...5094worker2 # [ 95.450658] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.047Z level=INFO msg="All 1 paths already cached"5095client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5096worker2 # [ 95.456394] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:603485097worker2 # [ 95.471593] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)5098worker2 # [ 95.622120] nix-daemon[856]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5099worker2 # [ 95.647190] nix-daemon[856]: reaped child process 1209, status = succeeded5100worker2 # [ 95.746444] nix-daemon[856]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5101worker2 # [ 95.768713] nix-daemon[856]: reaped child process 1214, status = succeeded5102lb # [ 95.771915] niks3-server[937]: 2026/09/22 15:46:48 INFO Received uploads request method=POST path=/api/pending_closures5103worker2 # [ 95.783365] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.382Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5104worker2 # [ 95.785051] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.382Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5105lb # [ 95.806121] niks3-server[937]: 2026/09/22 15:46:48 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5106worker2 # [ 95.823718] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.422Z level=INFO msg="Uploading 1 narinfos"5107lb # [ 95.824445] niks3-server[937]: 2026/09/22 15:46:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5108lb # [ 95.834605] niks3-server[937]: 2026/09/22 15:46:48 INFO Signed narinfos id=33 count=15109lb # [ 95.839796] niks3-server[937]: 2026/09/22 15:46:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5110worker2 # [ 95.852088] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.451Z level=INFO msg="Upload complete. (364ms)"5111lb # [ 95.852536] niks3-server[937]: 2026/09/22 15:46:48 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5112lb # [ 95.855333] niks3-server[937]: 2026/09/22 15:46:48 INFO Completed upload id=335113lb # [ 95.859076] niks3-server[937]: 2026/09/22 15:46:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5114worker2 # [ 95.860093] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=5685115worker2 # [ 95.868752] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603485116worker2 # [ 95.879587] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)5117worker2 # [ 95.884856] nix-daemon[856]: reaped child process 1204, status = succeeded5118client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5119worker2 # [ 95.897592] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted)5120client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5121client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5122worker2 # [ 97.206699] nix-daemon[856]: reaped child process 1218, status = succeeded5123worker2 # [ 97.280462] nix-daemon[856]: accepted connection from pid 1234, user nix-grpc-daemon (trusted)5124worker2 # [ 97.302312] nix-daemon[856]: reaped child process 1237, status = succeeded5125lb # [ 97.302033] niks3-server[937]: 2026/09/22 15:46:49 INFO Received uploads request method=POST path=/api/pending_closures5126worker2 # [ 97.314226] nix-grpc-daemon[910]: time=2026-09-22T15:46:49.911Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5127lb # [ 97.335591] niks3-server[937]: 2026/09/22 15:46:49 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5128worker2 # [ 97.346661] nix-grpc-daemon[910]: time=2026-09-22T15:46:49.945Z level=INFO msg="Uploading 1 narinfos"5129lb # [ 97.350452] niks3-server[937]: 2026/09/22 15:46:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5130lb # [ 97.358801] niks3-server[937]: 2026/09/22 15:46:49 INFO Signed narinfos id=34 count=15131lb # [ 97.371803] niks3-server[937]: 2026/09/22 15:46:49 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5132lb # [ 97.379518] niks3-server[937]: 2026/09/22 15:46:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5133lb # [ 97.383481] niks3-server[937]: 2026/09/22 15:46:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5134lb # [ 97.388253] niks3-server[937]: 2026/09/22 15:46:49 INFO Completed upload id=345135worker2 # [ 97.388600] nix-grpc-daemon[910]: time=2026-09-22T15:46:49.987Z level=INFO msg="Upload complete. (280ms)"5136worker2 # [ 97.401250] nix-grpc-daemon[772]: ts=2026-09-22T15:46:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15137worker2 # [ 97.418948] nix-daemon[856]: reaped child process 1216, status = succeeded5138client # copying 1 paths...5139worker2 # [ 97.446954] nix-grpc-daemon[772]: ts=2026-09-22T15:46:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:603485140client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5141worker1 # [ 97.505917] nix-grpc-daemon[765]: ts=2026-09-22T15:46:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:416765142worker1 # [ 97.518765] nix-grpc-daemon[765]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5143worker1 # [ 97.535509] nix-grpc-daemon[765]: ts=2026-09-22T15:46:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=1205144client # [ 97.640713] nix-daemon[1089]: reaped child process 1218, status = succeeded5145client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.59 seconds)5146(finished: subtest: an input only one worker has still reaches the builder, in 10.59 seconds)5147subtest: upload whose reference the worker lacks is completed from the cache5148client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5149client # this derivation will be built:5150client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5151client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5152client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5153client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5154client # this derivation will be built:5155client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5156client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5157client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)5158client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5159client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5160worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5161lb # [ 98.536550] envoy[894]: conn peer=[2001:db8:1::1]:37746 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=685162lb # [ 98.541178] envoy[894]: conn peer=[2001:db8:1::1]:51854 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4267 tx=2117 ms=31285163lb # [ 98.545981] envoy[894]: conn peer=[2001:db8:1::1]:37748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3773 tx=2142 ms=30025164lb # [ 98.550776] envoy[894]: conn peer=[2001:db8:1::1]:37752 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=305165lb # [ 98.555423] envoy[894]: conn peer=[2001:db8:1::1]:37762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3756 tx=2125 ms=22005166lb # [ 98.560143] envoy[894]: conn peer=[2001:db8:1::1]:37774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=945167worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.21 seconds)5168worker2 # [ 98.663348] systemd[1]: Stopping Nix worker-protocol over gRPC...5169worker2 # [ 98.858032] nix-grpc-daemon[772]: ts=2026-09-22T15:46:51Z level=info event=signal_exit5170lb # [ 99.825986] niks3-server[937]: 2026/09/22 15:46:52 INFO lead: released remote=192.168.1.4:571285171lb # [ 100.809692] niks3-server[937]: 2026/09/22 15:46:53 INFO lead: acquired remote=192.168.1.3:354145172worker1 # [ 100.837730] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=scheduler_take_over5173worker1 # [ 100.853714] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=scheduler_restarting addr=lb:500515174worker2 # [ 100.829946] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=scheduler_restarting5175worker2 # [ 100.830341] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=scheduler_restarting addr=lb:500515176worker2 # [ 100.830468] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:603365177worker2 # [ 100.830583] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=worker_gone addr=192.168.1.4:500515178worker2 # [ 100.850307] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:603365179worker2 # [ 100.855281] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=worker_gone addr=192.168.1.3:500515180worker2 # [ 100.886373] nix-daemon[856]: reaped child process 858, status = succeeded5181worker1 # [ 101.055098] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=scheduler_connected addr=lb:500515182worker2 # [ 101.027353] nix-grpc-daemon[772]: E0000 00:00:1790092013.626389 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5183worker1 # [ 101.075496] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:406845184worker1 # [ 101.081726] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05185worker2 # [ 101.060661] nix-grpc-daemon[772]: E0000 00:00:1790092013.626619 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5186worker2 # [ 101.067322] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5187worker2 # [ 102.944570] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5188worker2 # [ 102.951921] systemd[1]: Stopped Nix worker-protocol over gRPC.5189worker2 # [ 102.954762] systemd[1]: nix-grpc-daemon.service: Consumed 3.395s CPU time over 1min 24.305s wall clock time, 63.2M memory peak, 87.2K incoming IP traffic, 211.8K outgoing IP traffic.5190worker2 # [ 102.970539] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5191worker2 # [ 102.978460] systemd[1]: Closed Nix worker-protocol over gRPC.5192worker2 # [ 102.981015] systemd[1]: nix-grpc-daemon.socket: Consumed 712us CPU time over 1min 32.699s wall clock time, 256K memory peak, 113.2K incoming IP traffic, 95.9K outgoing IP traffic.5193lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5194lb: must succeed: curl -sf localhost:9901/clusters5195lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5196lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5197client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25198worker1 # [ 103.315595] nix-grpc-daemon[765]: ts=2026-09-22T15:46:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:416765199worker1 # [ 103.324283] nix-grpc-daemon[765]: ts=2026-09-22T15:46:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:416765200client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5201client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.26 seconds)5202worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5203worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5204worker2 # [ 103.422857] systemd[1]: Listening on Nix worker-protocol over gRPC.5205worker1 # [ 103.520963] systemd[1]: Stopping Nix worker-protocol over gRPC...5206worker1 # [ 103.827486] nix-grpc-daemon[765]: ts=2026-09-22T15:46:56Z level=info event=signal_exit5207lb # [ 103.818020] niks3-server[937]: 2026/09/22 15:46:56 INFO lead: released remote=192.168.1.3:354145208worker2 # [ 103.943143] systemd[1]: Starting scheduler-token.service...5209worker2 # [ 104.031818] systemd[1]: scheduler-token.service: Deactivated successfully.5210worker2 # [ 104.041538] systemd[1]: Finished scheduler-token.service.5211worker2 # [ 104.043515] systemd[1]: scheduler-token.service: Consumed 59ms CPU time over 92ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5212worker2 # [ 104.056591] systemd[1]: Starting Nix worker-protocol over gRPC...5213worker2 # [ 104.194344] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5214worker2 # [ 104.196695] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5215worker2 # [ 104.200618] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15216worker2 # [ 104.207535] nix-grpc-daemon[1261]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5217worker2 # [ 104.221319] nix-grpc-daemon[1261]: E0000 00:00:1790092016.804547 1261 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5218lb # [ 104.224833] niks3-server[937]: 2026/09/22 15:46:56 INFO lead: acquired remote=192.168.1.4:338285219worker2 # [ 104.228109] nix-grpc-daemon[1261]: E0000 00:00:1790092016.806767 1261 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5220worker2 # [ 104.256164] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5221worker2 # [ 104.257858] nix-grpc-daemon[1261]: E0000 00:00:1790092016.823616 1300 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5222worker2 # [ 104.263571] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=scheduler_take_over5223worker2 # [ 104.265208] nix-grpc-daemon[1261]: E0000 00:00:1790092016.843302 1300 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5224worker2 # [ 104.267647] nix-grpc-daemon[1261]: E0000 00:00:1790092016.843486 1300 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5225worker2 # [ 104.273000] systemd[1]: Started Nix worker-protocol over gRPC.5226worker2 # [ 104.275875] nix-grpc-daemon[1261]: E0000 00:00:1790092016.862074 1300 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5227worker2 # [ 104.278539] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=scheduler_connected addr=lb:500515228worker2 # [ 104.279812] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5229worker2 # [ 104.477437] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:500515230worker2 # [ 104.482247] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5231worker2 # [ 104.683395] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:500515232worker2 # [ 104.688818] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5233worker1 # [ 104.852310] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=scheduler_restarting5234worker1 # [ 104.856242] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=scheduler_restarting addr=lb:500515235worker1 # [ 104.860302] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:406845236worker1 # [ 104.868125] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=worker_gone addr=192.168.1.3:500515237worker1 # [ 104.896115] nix-daemon[829]: reaped child process 831, status = succeeded5238worker2 # [ 104.889883] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:500515239worker2 # [ 104.894931] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5240worker1 # [ 104.992345] nix-grpc-daemon[765]: E0000 00:00:1790092017.594927 810 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5241worker1 # [ 104.998937] nix-grpc-daemon[765]: E0000 00:00:1790092017.595204 810 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5242worker1 # [ 105.069377] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5243worker2 # [ 105.096001] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:500515244worker2 # [ 105.100996] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5245worker2 # [ 105.304998] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:500515246worker2 # [ 105.319185] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=oidc_jwks provider=mock keys=15247worker2 # [ 105.323817] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:491325248worker2 # [ 105.329493] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05249worker1 # [ 106.905877] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5250worker1 # [ 106.910901] systemd[1]: Stopped Nix worker-protocol over gRPC.5251worker1 # [ 106.916482] systemd[1]: nix-grpc-daemon.service: Consumed 4.744s CPU time over 1min 28.339s wall clock time, 65.1M memory peak, 102.2K incoming IP traffic, 182.6K outgoing IP traffic.5252worker1 # [ 106.932263] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5253worker1 # [ 106.938977] systemd[1]: Closed Nix worker-protocol over gRPC.5254worker1 # [ 106.941570] systemd[1]: nix-grpc-daemon.socket: Consumed 562us CPU time over 1min 36.769s wall clock time, 256K memory peak, 109.1K incoming IP traffic, 97.8K outgoing IP traffic.5255lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5256lb: must succeed: curl -sf localhost:9901/clusters5257lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5258lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5259client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25260worker2 # [ 107.223424] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491405261worker2 # [ 107.241190] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5262worker2 # [ 107.267865] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5)5263worker2 # [ 107.361523] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5)5264worker2 # [ 107.515644] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 3/5)5265worker2 # [ 107.562238] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 172 ms (attempt 4/5)5266worker2 # [ 107.737224] nix-grpc-daemon[1261]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5267worker2 # [ 107.743851] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5268client # copying 2 paths...5269client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5270worker2 # [ 107.767311] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491405271client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5272worker2 # [ 107.783835] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5273worker2 # [ 107.812731] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:00Z level=info event=niks3_push_started pid=13215274worker2 # [ 107.962497] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.561Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5275worker2 # [ 108.074653] nix-daemon[856]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5276worker2 # [ 108.094495] nix-daemon[856]: reaped child process 1337, status = succeeded5277lb # [ 108.099119] niks3-server[937]: 2026/09/22 15:47:00 INFO Received uploads request method=POST path=/api/pending_closures5278lb # [ 108.109702] niks3-server[937]: 2026/09/22 15:47:00 INFO Received uploads request method=POST path=/api/pending_closures5279worker2 # [ 108.121396] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.718Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5280worker2 # [ 108.122935] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.718Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5281worker2 # [ 108.126426] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.719Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5282lb # [ 108.157580] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5283lb # [ 108.171216] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5284lb # [ 108.184417] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5285lb # [ 108.195315] niks3-server[937]: 2026/09/22 15:47:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5286lb # [ 108.208368] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5287lb # [ 108.224192] niks3-server[937]: 2026/09/22 15:47:00 INFO Signed narinfos id=36 count=25288lb # [ 108.227933] niks3-server[937]: 2026/09/22 15:47:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5289worker2 # [ 108.230255] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.827Z level=INFO msg="Uploading 3 narinfos"5290lb # [ 108.235203] niks3-server[937]: 2026/09/22 15:47:00 INFO Signed narinfos id=35 count=15291lb # [ 108.258739] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5292lb # [ 108.271457] niks3-server[937]: 2026/09/22 15:47:00 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5293lb # [ 108.279455] niks3-server[937]: 2026/09/22 15:47:00 INFO Completed upload id=355294lb # [ 108.284806] niks3-server[937]: 2026/09/22 15:47:00 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5295lb # [ 108.286835] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5296worker2 # [ 108.293994] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.893Z level=INFO msg="Upload complete. (327ms)"5297lb # [ 108.296874] niks3-server[937]: 2026/09/22 15:47:00 INFO Completed upload id=365298lb # [ 108.299132] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5299worker2 # [ 108.308662] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 paths=2 nar_bytes_in=3125300worker2 # [ 108.325714] nix-daemon[856]: reaped child process 1319, status = succeeded5301client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.34 seconds)5302worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5303worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5304worker1 # [ 108.473170] systemd[1]: Listening on Nix worker-protocol over gRPC.5305worker1 # [ 108.482093] systemd[1]: Starting Nix worker-protocol over gRPC...5306lb # [ 108.538184] envoy[894]: conn peer=[2001:db8:1::4]:46336 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=47244 tx=2772 ms=821415307lb # [ 108.543579] envoy[894]: conn peer=[2001:db8:1::1]:40618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=295308lb # [ 108.548428] envoy[894]: conn peer=[2001:db8:1::3]:37948 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10545 tx=3082 ms=861965309lb # [ 108.553346] envoy[894]: conn peer=[2001:db8:1::1]:40624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1009 tx=282 ms=10995310worker1 # [ 108.627045] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5311worker1 # [ 108.629465] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5312worker1 # [ 108.633514] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15313worker1 # [ 108.638957] nix-grpc-daemon[1301]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5314worker1 # [ 108.656260] nix-grpc-daemon[1301]: E0000 00:00:1790092021.241318 1301 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5315lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5316lb: must succeed: curl -sf localhost:9901/clusters5317worker1 # [ 108.688327] nix-grpc-daemon[1301]: E0000 00:00:1790092021.244962 1301 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5318worker1 # [ 108.691206] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5319worker1 # [ 108.693183] nix-grpc-daemon[1301]: E0000 00:00:1790092021.253710 1342 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5320worker1 # [ 108.695520] nix-grpc-daemon[1301]: E0000 00:00:1790092021.273597 1342 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5321worker1 # [ 108.697936] nix-grpc-daemon[1301]: E0000 00:00:1790092021.273790 1342 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5322worker1 # [ 108.703485] nix-grpc-daemon[1301]: E0000 00:00:1790092021.290273 1342 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5323worker1 # [ 108.707882] systemd[1]: Started Nix worker-protocol over gRPC.5324worker1 # [ 108.708954] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=scheduler_connected addr=lb:500515325worker2 # [ 108.683880] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:491325326worker2 # [ 108.690084] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05327lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5328lb: must succeed: curl -sf localhost:9901/clusters5329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5330lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5331(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.15 seconds)5332subtest: a niks3 restart does not cost the leader its role5333lb: must succeed: curl -sf localhost:9901/clusters5334lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5335lb: must succeed: curl -sf localhost:9901/clusters5336lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5337lb # [ 109.923794] systemd[1]: Stopping niks3 server...5338worker1 # [ 109.951277] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:02Z level=info event=scheduler_lock_lost status=2005339lb # [ 109.926823] niks3-server[937]: 2026/09/22 15:47:02 INFO Shutdown signal received, draining in-flight requests timeout=10s5340worker2 # [ 109.926842] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:02Z level=info event=scheduler_lock_lost status=2005341lb # [ 109.926932] niks3-server[937]: 2026/09/22 15:47:02 INFO lead: released remote=192.168.1.4:338285342lb # [ 109.950791] systemd[1]: niks3.service: Deactivated successfully.5343lb # [ 109.952511] systemd[1]: Stopped niks3 server.5344lb # [ 109.954330] systemd[1]: niks3.service: Consumed 665ms CPU time over 1min 29.376s wall clock time, 27.4M memory peak, 96.5K incoming IP traffic, 139.1K outgoing IP traffic.5345lb # [ 109.965029] systemd[1]: Starting niks3 server...5346lb # [ 110.030424] niks3-server[1077]: 2026/09/22 15:47:02 goose: no migrations to run. current version: 202609200000005347lb # [ 110.035328] niks3-server[1077]: 2026/09/22 15:47:02 OK 1_commit_pending_closure.sql (6.34ms)5348lb # [ 110.039354] niks3-server[1077]: 2026/09/22 15:47:02 OK 2_object_stats_trigger.sql (3.67ms)5349lb # [ 110.041236] niks3-server[1077]: 2026/09/22 15:47:02 goose: up to current file version: 25350lb # [ 110.043176] niks3-server[1077]: 2026/09/22 15:47:02 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5351lb # [ 110.053137] niks3-server[1077]: 2026/09/22 15:47:02 INFO Read proxy enabled — serving cache objects from S35352lb # [ 110.054444] niks3-server[1077]: 2026/09/22 15:47:02 INFO Using socket-activated listener address=0.0.0.0:57515353lb # [ 110.059697] niks3-server[1077]: 2026/09/22 15:47:02 INFO systemd watchdog enabled interval=15s5354lb # [ 110.061576] niks3-server[1077]: 2026/09/22 15:47:02 INFO Starting HTTP server address=0.0.0.0:57515355lb # [ 110.065278] systemd[1]: Started niks3 server.5356lb: waiting for unit niks3.service5357lb: (finished: waiting for unit niks3.service, in 0.04 seconds)5358lb: must succeed: curl -sf localhost:9901/clusters5359lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5360worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5361worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5362lb: must succeed: curl -sf localhost:9901/clusters5363lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5364lb: must succeed: curl -sf localhost:9901/clusters5365lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5366worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5367worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5368client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25369worker2 # [ 110.561063] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:491405370client # don't know how to build these paths:5371client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5372worker2 # [ 110.573496] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491405373worker2 # [ 110.604137] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05374worker2 # [ 110.610304] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15375worker2 # [ 110.656581] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491405376worker2 # [ 110.672113] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491405377client # copying 1 paths to 'lb:50051'...5378client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5379worker2 # [ 110.687759] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491405380worker2 # [ 110.703030] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5381worker2 # [ 110.865844] nix-daemon[856]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5382worker2 # [ 110.885910] nix-daemon[856]: reaped child process 1357, status = succeeded5383lb # [ 110.930222] niks3-server[1077]: 2026/09/22 15:47:03 INFO lead: acquired remote=192.168.1.4:551425384worker2 # [ 110.989409] nix-daemon[856]: accepted connection from pid 1359, user nix-grpc-daemon (trusted)5385worker2 # [ 111.008689] nix-daemon[856]: reaped child process 1364, status = succeeded5386lb # [ 111.006403] niks3-server[1077]: 2026/09/22 15:47:03 INFO Received uploads request method=POST path=/api/pending_closures5387worker2 # [ 111.036169] nix-grpc-daemon[1321]: time=2026-09-22T15:47:03.634Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5388worker2 # [ 111.038099] nix-grpc-daemon[1321]: time=2026-09-22T15:47:03.634Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5389lb # [ 111.076810] niks3-server[1077]: 2026/09/22 15:47:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5390lb # [ 111.092256] niks3-server[1077]: 2026/09/22 15:47:03 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5391lb # [ 111.112621] niks3-server[1077]: 2026/09/22 15:47:03 INFO Signed narinfos id=37 count=15392lb # [ 111.117740] niks3-server[1077]: 2026/09/22 15:47:03 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5393worker2 # [ 111.118894] nix-grpc-daemon[1321]: time=2026-09-22T15:47:03.717Z level=INFO msg="Uploading 1 narinfos"5394lb # [ 111.125674] niks3-server[1077]: 2026/09/22 15:47:03 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5395lb # [ 111.134788] niks3-server[1077]: 2026/09/22 15:47:03 INFO Completed upload id=375396worker2 # [ 111.135838] nix-grpc-daemon[1321]: time=2026-09-22T15:47:03.735Z level=INFO msg="Upload complete. (415ms)"5397lb # [ 111.139899] niks3-server[1077]: 2026/09/22 15:47:03 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5398worker2 # [ 111.148251] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 paths=1 nar_bytes_in=5445399worker2 # [ 111.160891] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491405400worker2 # [ 111.175555] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5401worker2 # [ 111.181888] nix-daemon[856]: reaped child process 1352, status = succeeded5402client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5403worker2 # [ 111.193607] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5404client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5405client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5406client # farm-a> LOG-farm-a-after-niks3-restart5407worker2 # [ 112.072084] nix-daemon[856]: reaped child process 1368, status = succeeded5408worker2 # [ 112.172787] nix-daemon[856]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)5409worker2 # [ 112.191214] nix-daemon[856]: reaped child process 1387, status = succeeded5410lb # [ 112.198600] niks3-server[1077]: 2026/09/22 15:47:04 INFO Received uploads request method=POST path=/api/pending_closures5411worker2 # [ 112.215128] nix-grpc-daemon[1321]: time=2026-09-22T15:47:04.811Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5412worker2 # [ 112.217300] nix-grpc-daemon[1321]: time=2026-09-22T15:47:04.811Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5413lb # [ 112.236649] niks3-server[1077]: 2026/09/22 15:47:04 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5414worker2 # [ 112.256072] nix-grpc-daemon[1321]: time=2026-09-22T15:47:04.855Z level=INFO msg="Uploading 1 narinfos"5415lb # [ 112.256721] niks3-server[1077]: 2026/09/22 15:47:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5416lb # [ 112.273795] niks3-server[1077]: 2026/09/22 15:47:04 INFO Signed narinfos id=38 count=15417lb # [ 112.280971] niks3-server[1077]: 2026/09/22 15:47:04 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5418lb # [ 112.290676] niks3-server[1077]: 2026/09/22 15:47:04 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5419lb # [ 112.302622] niks3-server[1077]: 2026/09/22 15:47:04 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5420worker2 # [ 112.303795] nix-grpc-daemon[1321]: time=2026-09-22T15:47:04.902Z level=INFO msg="Upload complete. (298ms)"5421lb # [ 112.308264] niks3-server[1077]: 2026/09/22 15:47:04 INFO Completed upload id=385422lb # [ 112.312688] niks3-server[1077]: 2026/09/22 15:47:04 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5423worker2 # [ 112.327730] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15424worker2 # [ 112.344417] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05425worker2 # [ 112.349839] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15426worker2 # [ 112.354901] nix-daemon[856]: reaped child process 1366, status = succeeded5427worker2 # [ 112.373670] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491405428worker2 # [ 112.390507] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491405429worker2 # [ 112.406587] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.005Z level=INFO msg="All 1 paths already cached"5430client # copying 1 paths to 'lb:50051'...5431client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5432worker2 # [ 112.417682] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491405433worker2 # [ 112.431577] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5434worker2 # [ 112.582457] nix-daemon[856]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5435worker2 # [ 112.606306] nix-daemon[856]: reaped child process 1396, status = succeeded5436worker2 # [ 112.703717] nix-daemon[856]: accepted connection from pid 1398, user nix-grpc-daemon (trusted)5437worker2 # [ 112.720479] nix-daemon[856]: reaped child process 1401, status = succeeded5438lb # [ 112.725619] niks3-server[1077]: 2026/09/22 15:47:05 INFO Received uploads request method=POST path=/api/pending_closures5439worker2 # [ 112.739009] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.338Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5440worker2 # [ 112.740711] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.338Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5441lb # [ 112.760381] niks3-server[1077]: 2026/09/22 15:47:05 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5442lb # [ 112.776915] niks3-server[1077]: 2026/09/22 15:47:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5443worker2 # [ 112.782780] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.381Z level=INFO msg="Uploading 1 narinfos"5444lb # [ 112.788264] niks3-server[1077]: 2026/09/22 15:47:05 INFO Signed narinfos id=39 count=15445lb # [ 112.793103] niks3-server[1077]: 2026/09/22 15:47:05 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5446lb # [ 112.802525] niks3-server[1077]: 2026/09/22 15:47:05 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5447lb # [ 112.809946] niks3-server[1077]: 2026/09/22 15:47:05 INFO Completed upload id=395448worker2 # [ 112.809951] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.409Z level=INFO msg="Upload complete. (362ms)"5449lb # [ 112.811999] niks3-server[1077]: 2026/09/22 15:47:05 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5450worker2 # [ 112.820090] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 paths=1 nar_bytes_in=6565451worker2 # [ 112.835344] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491405452worker2 # [ 112.841580] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5453worker2 # [ 112.846618] nix-daemon[856]: reaped child process 1391, status = succeeded5454worker2 # [ 112.856805] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5455client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5456client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5457client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5458client # farm-b> LOG-farm-b-after-niks3-restart5459worker2 # [ 114.344480] nix-daemon[856]: reaped child process 1405, status = succeeded5460worker2 # [ 114.442830] nix-daemon[856]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5461worker2 # [ 114.463288] nix-daemon[856]: reaped child process 1424, status = succeeded5462lb # [ 114.467256] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received uploads request method=POST path=/api/pending_closures5463worker2 # [ 114.478785] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.077Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5464worker2 # [ 114.481046] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.077Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5465lb # [ 114.506037] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5466lb # [ 114.519670] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5467worker2 # [ 114.531124] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.130Z level=INFO msg="Uploading 1 narinfos"5468lb # [ 114.532336] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5469lb # [ 114.535664] niks3-server[1077]: 2026/09/22 15:47:07 INFO Signed narinfos id=40 count=15470lb # [ 114.546347] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5471worker2 # [ 114.551955] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.151Z level=INFO msg="Upload complete. (283ms)"5472lb # [ 114.553270] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5473lb # [ 114.556073] niks3-server[1077]: 2026/09/22 15:47:07 INFO Completed upload id=405474lb # [ 114.558427] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5475worker2 # [ 114.572193] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15476worker2 # [ 114.588524] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05477worker2 # [ 114.593682] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15478worker2 # [ 114.598129] nix-daemon[856]: reaped child process 1403, status = succeeded5479worker2 # [ 114.617809] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491405480worker2 # [ 114.634146] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491405481worker2 # [ 114.648707] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.247Z level=INFO msg="All 2 paths already cached"5482client # copying 1 paths to 'lb:50051'...5483client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5484worker2 # [ 114.660316] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491405485worker2 # [ 114.673870] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5486worker2 # [ 114.836794] nix-daemon[856]: accepted connection from pid 1430, user nix-grpc-daemon (trusted)5487worker2 # [ 114.860947] nix-daemon[856]: reaped child process 1433, status = succeeded5488worker2 # [ 114.957174] nix-daemon[856]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5489worker2 # [ 114.976378] nix-daemon[856]: reaped child process 1438, status = succeeded5490lb # [ 114.974346] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received uploads request method=POST path=/api/pending_closures5491worker2 # [ 114.986273] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.585Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5492worker2 # [ 114.988566] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.587Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5493lb # [ 115.008165] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5494worker2 # [ 115.025175] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.624Z level=INFO msg="Uploading 1 narinfos"5495lb # [ 115.025075] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5496lb # [ 115.034807] niks3-server[1077]: 2026/09/22 15:47:07 INFO Signed narinfos id=41 count=15497lb # [ 115.041582] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5498lb # [ 115.054494] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5499worker2 # [ 115.062169] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.661Z level=INFO msg="Upload complete. (371ms)"5500lb # [ 115.062691] niks3-server[1077]: 2026/09/22 15:47:07 INFO Completed upload id=415501lb # [ 115.067991] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5502worker2 # [ 115.072336] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 paths=1 nar_bytes_in=7845503worker2 # [ 115.093045] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491405504worker2 # [ 115.100554] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5505worker2 # [ 115.107001] nix-daemon[856]: reaped child process 1428, status = succeeded5506client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5507worker2 # [ 115.120603] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5508client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5509client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5510client # farm-top> LOG-farm-top-after-niks3-restart5511worker2 # [ 115.800686] nix-daemon[856]: reaped child process 1442, status = succeeded5512worker2 # [ 115.907124] nix-daemon[856]: accepted connection from pid 1458, user nix-grpc-daemon (trusted)5513worker2 # [ 115.927285] nix-daemon[856]: reaped child process 1461, status = succeeded5514lb # [ 115.934014] niks3-server[1077]: 2026/09/22 15:47:08 INFO Received uploads request method=POST path=/api/pending_closures5515worker2 # [ 115.945833] nix-grpc-daemon[1321]: time=2026-09-22T15:47:08.543Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5516worker2 # [ 115.947543] nix-grpc-daemon[1321]: time=2026-09-22T15:47:08.543Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5517lb # [ 115.970398] niks3-server[1077]: 2026/09/22 15:47:08 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5518lb # [ 115.984411] niks3-server[1077]: 2026/09/22 15:47:08 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5519worker2 # [ 115.995325] nix-grpc-daemon[1321]: time=2026-09-22T15:47:08.594Z level=INFO msg="Uploading 1 narinfos"5520lb # [ 115.997762] niks3-server[1077]: 2026/09/22 15:47:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5521lb # [ 116.009992] niks3-server[1077]: 2026/09/22 15:47:08 INFO Signed narinfos id=42 count=15522lb # [ 116.018926] niks3-server[1077]: 2026/09/22 15:47:08 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5523lb # [ 116.033808] niks3-server[1077]: 2026/09/22 15:47:08 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5524lb # [ 116.038526] niks3-server[1077]: 2026/09/22 15:47:08 INFO Completed upload id=425525worker2 # [ 116.038130] nix-grpc-daemon[1321]: time=2026-09-22T15:47:08.637Z level=INFO msg="Upload complete. (305ms)"5526lb # [ 116.045028] niks3-server[1077]: 2026/09/22 15:47:08 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5527worker2 # [ 116.051290] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15528worker2 # [ 116.071975] nix-daemon[856]: reaped child process 1440, status = succeeded5529client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.75 seconds)5530client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5531client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5532(finished: subtest: a niks3 restart does not cost the leader its role, in 6.48 seconds)5533subtest: scheduler down: the other node takes the lock and keeps it5534lb: must succeed: curl -sf localhost:9901/clusters5535lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5536lb: must succeed: curl -sf localhost:9901/clusters5537lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5538lb: must succeed: curl -sf localhost:9901/clusters5539lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5540lb: must succeed: curl -sf localhost:9901/clusters5541lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5542worker2 # [ 116.524975] systemd[1]: Stopping Nix worker-protocol over gRPC...5543worker2 # [ 117.224820] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:09Z level=info event=signal_exit5544lb # [ 117.935958] niks3-server[1077]: 2026/09/22 15:47:10 INFO lead: released remote=192.168.1.4:551425545lb # [ 118.537446] envoy[894]: conn peer=[2001:db8:1::1]:40638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9766 tx=4492 ms=55095546worker1 # [ 118.963524] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_restarting addr=lb:500515547worker2 # [ 118.941845] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=scheduler_restarting5548worker2 # [ 118.948549] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=scheduler_restarting addr=lb:500515549worker2 # [ 118.953477] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:491325550worker2 # [ 118.959036] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=worker_gone addr=192.168.1.3:500515551worker2 # [ 118.963719] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:491325552worker2 # [ 118.969267] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=worker_gone addr=192.168.1.4:500515553worker2 # [ 118.992532] nix-daemon[856]: reaped child process 1313, status = succeeded5554worker1 # [ 119.163230] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_connected addr=lb:500515555worker1 # [ 119.168858] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5556worker2 # [ 119.157573] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5557worker2 # [ 119.277804] nix-grpc-daemon[1261]: E0000 00:00:1790092031.876705 1306 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5558worker2 # [ 119.284447] nix-grpc-daemon[1261]: E0000 00:00:1790092031.877014 1306 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5559worker1 # [ 119.369663] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_connected addr=lb:500515560worker1 # [ 119.374580] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5561worker1 # [ 119.575471] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:500515562worker1 # [ 119.580652] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5563worker1 # [ 119.781565] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:500515564worker1 # [ 119.786779] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5565worker1 # [ 119.989030] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:500515566worker1 # [ 119.993152] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5567worker1 # [ 120.189604] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:500515568worker1 # [ 120.194729] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5569worker2 # [ 120.244155] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5570worker2 # [ 120.250923] systemd[1]: Stopped Nix worker-protocol over gRPC.5571worker2 # [ 120.256731] systemd[1]: nix-grpc-daemon.service: Consumed 1.703s CPU time over 16.194s wall clock time, 38.1M memory peak, 45.6K incoming IP traffic, 81.4K outgoing IP traffic.5572worker2 # [ 120.268471] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5573worker2 # [ 120.277084] systemd[1]: Closed Nix worker-protocol over gRPC.5574worker2 # [ 120.278886] systemd[1]: nix-grpc-daemon.socket: Consumed 906us CPU time over 16.858s wall clock time, 256K memory peak, 37.8K incoming IP traffic, 28.1K outgoing IP traffic.5575worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5576worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5577worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5578worker1 # [ 120.395398] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:500515579worker1 # [ 120.402498] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5580worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5581worker1 # [ 120.600362] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_connected addr=lb:500515582worker1 # [ 120.605633] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5583worker1 # [ 120.806308] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_connected addr=lb:500515584worker1 # [ 120.811564] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5585worker1 # [ 120.957030] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_take_over5586lb # [ 120.931483] niks3-server[1077]: 2026/09/22 15:47:13 INFO lead: acquired remote=192.168.1.3:474865587worker1 # [ 121.014004] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_connected addr=lb:500515588worker1 # [ 121.018125] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5589worker1 # [ 121.214671] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_connected addr=lb:500515590worker1 # [ 121.220059] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5591worker1 # [ 121.420763] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_connected addr=lb:500515592worker1 # [ 121.425968] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5593worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5594worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5595worker1 # [ 121.626813] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_connected addr=lb:500515596worker1 # [ 121.632247] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5597worker1 # [ 121.833159] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_connected addr=lb:500515598worker1 # [ 121.838348] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5599worker1 # [ 122.042482] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_connected addr=lb:500515600worker1 # [ 122.049620] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:358625601worker1 # [ 122.054543] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05602worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5603worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5604client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25605worker1 # [ 122.795340] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:358705606worker1 # [ 122.810773] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5607client # don't know how to build these paths:5608client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5609worker1 # [ 122.836406] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358705610worker1 # [ 122.852329] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5)5611worker1 # [ 122.924773] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 2/5)5612worker1 # [ 123.055024] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 3/5)5613worker1 # [ 123.067662] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 4/5)5614worker1 # [ 123.197626] nix-grpc-daemon[1301]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5615worker1 # [ 123.204918] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5616worker1 # [ 123.235155] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05617worker1 # [ 123.242716] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15618worker1 # [ 123.283744] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358705619worker1 # [ 123.299494] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358705620client # copying 1 paths to 'lb:50051'...5621client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5622worker1 # [ 123.316433] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358705623worker1 # [ 123.331172] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5624worker1 # [ 123.368290] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=info event=niks3_push_started pid=13805625worker1 # [ 123.519913] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.122Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5626worker1 # [ 123.638218] nix-daemon[829]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)5627worker1 # [ 123.658288] nix-daemon[829]: reaped child process 1395, status = succeeded5628worker1 # [ 123.764345] nix-daemon[829]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)5629worker1 # [ 123.783782] nix-daemon[829]: reaped child process 1400, status = succeeded5630lb # [ 123.762027] niks3-server[1077]: 2026/09/22 15:47:16 INFO Received uploads request method=POST path=/api/pending_closures5631worker1 # [ 123.800072] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.402Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5632worker1 # [ 123.801610] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.402Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5633lb # [ 123.799046] niks3-server[1077]: 2026/09/22 15:47:16 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5634worker1 # [ 123.839186] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.441Z level=INFO msg="Uploading 1 narinfos"5635lb # [ 123.813668] niks3-server[1077]: 2026/09/22 15:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5636lb # [ 123.823041] niks3-server[1077]: 2026/09/22 15:47:16 INFO Signed narinfos id=43 count=15637lb # [ 123.830089] niks3-server[1077]: 2026/09/22 15:47:16 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5638lb # [ 123.844656] niks3-server[1077]: 2026/09/22 15:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5639lb # [ 123.854346] niks3-server[1077]: 2026/09/22 15:47:16 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5640lb # [ 123.860453] niks3-server[1077]: 2026/09/22 15:47:16 INFO Completed upload id=435641worker1 # [ 123.886761] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.489Z level=INFO msg="Upload complete. (363ms)"5642worker1 # [ 123.896872] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=0 paths=1 nar_bytes_in=5205643worker1 # [ 123.908674] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358705644worker1 # [ 123.920401] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5645worker1 # [ 123.925487] nix-daemon[829]: reaped child process 1378, status = succeeded5646client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5647worker1 # [ 123.937612] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5648client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5649client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5650client # farm-a> LOG-farm-a-on-standby5651worker1 # [ 125.625697] nix-daemon[829]: reaped child process 1404, status = succeeded5652worker1 # [ 125.722834] nix-daemon[829]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)5653worker1 # [ 125.740038] nix-daemon[829]: reaped child process 1423, status = succeeded5654lb # [ 125.719942] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received uploads request method=POST path=/api/pending_closures5655worker1 # [ 125.759446] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.359Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5656worker1 # [ 125.762221] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.359Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5657lb # [ 125.761183] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5658lb # [ 125.772675] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5659worker1 # [ 125.813801] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.415Z level=INFO msg="Uploading 1 narinfos"5660lb # [ 125.788129] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5661lb # [ 125.796999] niks3-server[1077]: 2026/09/22 15:47:18 INFO Signed narinfos id=44 count=15662lb # [ 125.803526] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5663lb # [ 125.812312] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5664lb # [ 125.821833] niks3-server[1077]: 2026/09/22 15:47:18 INFO Completed upload id=445665worker1 # [ 125.847423] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.450Z level=INFO msg="Upload complete. (299ms)"5666lb # [ 125.827413] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5667worker1 # [ 125.870883] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15668worker1 # [ 125.887638] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05669worker1 # [ 125.891741] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15670worker1 # [ 125.896069] nix-daemon[829]: reaped child process 1402, status = succeeded5671worker1 # [ 125.914906] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358705672worker1 # [ 125.929785] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358705673worker1 # [ 125.944391] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.547Z level=INFO msg="All 1 paths already cached"5674client # copying 1 paths to 'lb:50051'...5675client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5676worker1 # [ 125.954827] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358705677worker1 # [ 125.968818] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5678worker1 # [ 126.118650] nix-daemon[829]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5679worker1 # [ 126.137342] nix-daemon[829]: reaped child process 1432, status = succeeded5680worker1 # [ 126.236637] nix-daemon[829]: accepted connection from pid 1434, user nix-grpc-daemon (trusted)5681worker1 # [ 126.252553] nix-daemon[829]: reaped child process 1437, status = succeeded5682lb # [ 126.231791] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received uploads request method=POST path=/api/pending_closures5683worker1 # [ 126.268090] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.870Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5684worker1 # [ 126.269550] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.870Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5685lb # [ 126.263701] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5686lb # [ 126.280327] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5687worker1 # [ 126.312169] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.914Z level=INFO msg="Uploading 1 narinfos"5688lb # [ 126.291735] niks3-server[1077]: 2026/09/22 15:47:18 INFO Signed narinfos id=45 count=15689lb # [ 126.297550] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5690lb # [ 126.308384] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5691worker1 # [ 126.339758] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.942Z level=INFO msg="Upload complete. (355ms)"5692lb # [ 126.314462] niks3-server[1077]: 2026/09/22 15:47:18 INFO Completed upload id=455693lb # [ 126.317566] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5694worker1 # [ 126.348282] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=0 paths=1 nar_bytes_in=6405695worker1 # [ 126.363178] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358705696worker1 # [ 126.369830] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5697worker1 # [ 126.374387] nix-daemon[829]: reaped child process 1427, status = succeeded5698client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5699worker1 # [ 126.386975] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5700client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5701client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5702client # farm-b> LOG-farm-b-on-standby5703worker1 # [ 127.652467] nix-daemon[829]: reaped child process 1441, status = succeeded5704worker1 # [ 127.782104] nix-daemon[829]: accepted connection from pid 1457, user nix-grpc-daemon (trusted)5705worker1 # [ 127.803708] nix-daemon[829]: reaped child process 1460, status = succeeded5706lb # [ 127.783062] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received uploads request method=POST path=/api/pending_closures5707worker1 # [ 127.819848] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.421Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5708worker1 # [ 127.821823] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.421Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5709lb # [ 127.819755] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5710lb # [ 127.832998] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5711worker1 # [ 127.872296] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.474Z level=INFO msg="Uploading 1 narinfos"5712lb # [ 127.847819] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5713lb # [ 127.857291] niks3-server[1077]: 2026/09/22 15:47:20 INFO Signed narinfos id=46 count=15714lb # [ 127.863174] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5715worker1 # [ 127.900393] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.501Z level=INFO msg="Upload complete. (300ms)"5716lb # [ 127.873733] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5717lb # [ 127.878042] niks3-server[1077]: 2026/09/22 15:47:20 INFO Completed upload id=465718lb # [ 127.881262] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5719worker1 # [ 127.913114] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15720worker1 # [ 127.934236] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05721worker1 # [ 127.940594] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15722worker1 # [ 127.944712] nix-daemon[829]: reaped child process 1439, status = succeeded5723worker1 # [ 127.978867] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358705724worker1 # [ 127.994936] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358705725worker1 # [ 128.010652] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.613Z level=INFO msg="All 2 paths already cached"5726client # copying 1 paths to 'lb:50051'...5727client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5728worker1 # [ 128.022060] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358705729worker1 # [ 128.036417] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5730worker1 # [ 128.200616] nix-daemon[829]: accepted connection from pid 1466, user nix-grpc-daemon (trusted)5731worker1 # [ 128.230237] nix-daemon[829]: reaped child process 1469, status = succeeded5732worker1 # [ 128.330203] nix-daemon[829]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5733worker1 # [ 128.351658] nix-daemon[829]: reaped child process 1474, status = succeeded5734lb # [ 128.329029] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received uploads request method=POST path=/api/pending_closures5735worker1 # [ 128.364170] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.967Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5736worker1 # [ 128.366160] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.968Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5737lb # [ 128.357232] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5738worker1 # [ 128.398137] nix-grpc-daemon[1380]: time=2026-09-22T15:47:21.000Z level=INFO msg="Uploading 1 narinfos"5739lb # [ 128.371509] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5740lb # [ 128.382730] niks3-server[1077]: 2026/09/22 15:47:20 INFO Signed narinfos id=47 count=15741lb # [ 128.390471] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5742lb # [ 128.403878] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5743worker1 # [ 128.436814] nix-grpc-daemon[1380]: time=2026-09-22T15:47:21.039Z level=INFO msg="Upload complete. (381ms)"5744lb # [ 128.411604] niks3-server[1077]: 2026/09/22 15:47:21 INFO Completed upload id=475745lb # [ 128.414047] niks3-server[1077]: 2026/09/22 15:47:21 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5746worker1 # [ 128.444498] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=0 paths=1 nar_bytes_in=7685747worker1 # [ 128.452609] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358705748worker1 # [ 128.462616] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5749worker1 # [ 128.464270] nix-daemon[829]: reaped child process 1464, status = succeeded5750client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5751worker1 # [ 128.474582] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5752client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5753lb # [ 128.538695] envoy[894]: conn peer=[2001:db8:1::4]:45714 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5496 tx=648 ms=148735754client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5755client # farm-top> LOG-farm-top-on-standby5756worker1 # [ 129.376546] nix-daemon[829]: reaped child process 1478, status = succeeded5757worker1 # [ 129.448752] nix-daemon[829]: accepted connection from pid 1494, user nix-grpc-daemon (trusted)5758worker1 # [ 129.469182] nix-daemon[829]: reaped child process 1497, status = succeeded5759lb # [ 129.447242] niks3-server[1077]: 2026/09/22 15:47:22 INFO Received uploads request method=POST path=/api/pending_closures5760worker1 # [ 129.484462] nix-grpc-daemon[1380]: time=2026-09-22T15:47:22.086Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5761worker1 # [ 129.486299] nix-grpc-daemon[1380]: time=2026-09-22T15:47:22.086Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5762lb # [ 129.483023] niks3-server[1077]: 2026/09/22 15:47:22 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5763lb # [ 129.497421] niks3-server[1077]: 2026/09/22 15:47:22 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5764worker1 # [ 129.538504] nix-grpc-daemon[1380]: time=2026-09-22T15:47:22.140Z level=INFO msg="Uploading 1 narinfos"5765lb # [ 129.512435] niks3-server[1077]: 2026/09/22 15:47:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5766lb # [ 129.523417] niks3-server[1077]: 2026/09/22 15:47:22 INFO Signed narinfos id=48 count=15767lb # [ 129.531044] niks3-server[1077]: 2026/09/22 15:47:22 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5768worker1 # [ 129.570463] nix-grpc-daemon[1380]: time=2026-09-22T15:47:22.173Z level=INFO msg="Upload complete. (287ms)"5769lb # [ 129.543942] niks3-server[1077]: 2026/09/22 15:47:22 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5770lb # [ 129.554138] niks3-server[1077]: 2026/09/22 15:47:22 INFO Completed upload id=485771lb # [ 129.557932] niks3-server[1077]: 2026/09/22 15:47:22 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5772worker1 # [ 129.585226] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15773worker1 # [ 129.603499] nix-daemon[829]: reaped child process 1476, status = succeeded5774client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 7.06 seconds)5775client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5776client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds)5777worker2 # [ 129.867111] systemd[1]: Listening on Nix worker-protocol over gRPC.5778worker2 # [ 129.877274] systemd[1]: Starting scheduler-token.service...5779worker2 # [ 129.956094] systemd[1]: scheduler-token.service: Deactivated successfully.5780worker2 # [ 129.963946] systemd[1]: Finished scheduler-token.service.5781worker2 # [ 129.966547] systemd[1]: scheduler-token.service: Consumed 44ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5782worker2 # [ 129.982305] systemd[1]: Starting Nix worker-protocol over gRPC...5783worker2 # [ 130.090064] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5784worker2 # [ 130.092348] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5785worker2 # [ 130.096325] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15786worker2 # [ 130.101624] nix-grpc-daemon[1482]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5787worker2 # [ 130.103094] nix-grpc-daemon[1482]: E0000 00:00:1790092042.700198 1482 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5788lb: must succeed: curl -sf localhost:9901/clusters5789worker2 # [ 130.134946] nix-grpc-daemon[1482]: E0000 00:00:1790092042.706403 1482 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5790worker2 # [ 130.150747] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5791worker2 # [ 130.153108] nix-grpc-daemon[1482]: E0000 00:00:1790092042.716097 1524 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5792worker2 # [ 130.155741] nix-grpc-daemon[1482]: E0000 00:00:1790092042.731552 1524 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5793worker2 # [ 130.161228] nix-grpc-daemon[1482]: E0000 00:00:1790092042.731716 1524 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5794worker2 # [ 130.165922] systemd[1]: Started Nix worker-protocol over gRPC.5795worker2 # [ 130.166773] nix-grpc-daemon[1482]: E0000 00:00:1790092042.749123 1524 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5796worker2 # [ 130.169577] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=scheduler_connected addr=lb:500515797worker1 # [ 130.202735] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:22Z level=info event=oidc_jwks provider=mock keys=15798worker1 # [ 130.207252] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:358625799worker1 # [ 130.212653] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05800lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5801worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5802worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5803lb: must succeed: curl -sf localhost:9901/clusters5804lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5805lb: must succeed: curl -sf localhost:9901/clusters5806lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5807worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5808worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5809(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.15 seconds)5810subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5811client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5812client # [ 130.502563] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5813client # Running as unit: rs.service; invocation ID: 6dda1e013a664e6ca11b449c6f50c90d5814client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5815worker2 # [ 130.700661] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504805816worker2 # [ 130.715128] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)5817client # [ 130.739721] nix[1342]: don't know how to build these paths:5818client # [ 130.742144] nix[1342]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5819worker1 # [ 130.769911] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05820worker1 # [ 130.777537] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15821worker1 # [ 130.823271] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358705822worker1 # [ 130.839046] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358705823client # [ 130.832236] nix[1342]: copying 1 paths to 'lb:50051'...5824client # [ 130.837732] nix[1342]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5825worker1 # [ 130.856305] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358705826worker1 # [ 130.870994] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5827worker1 # [ 131.036157] nix-daemon[829]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5828worker1 # [ 131.055139] nix-daemon[829]: reaped child process 1513, status = succeeded5829worker1 # [ 131.154639] nix-daemon[829]: accepted connection from pid 1515, user nix-grpc-daemon (trusted)5830worker1 # [ 131.170061] nix-daemon[829]: reaped child process 1518, status = succeeded5831lb # [ 131.151335] niks3-server[1077]: 2026/09/22 15:47:23 INFO Received uploads request method=POST path=/api/pending_closures5832worker1 # [ 131.188636] nix-grpc-daemon[1380]: time=2026-09-22T15:47:23.791Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5833worker1 # [ 131.190108] nix-grpc-daemon[1380]: time=2026-09-22T15:47:23.791Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5834lb # [ 131.185851] niks3-server[1077]: 2026/09/22 15:47:23 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5835worker1 # [ 131.223855] nix-grpc-daemon[1380]: time=2026-09-22T15:47:23.826Z level=INFO msg="Uploading 1 narinfos"5836lb # [ 131.199520] niks3-server[1077]: 2026/09/22 15:47:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5837lb # [ 131.208536] niks3-server[1077]: 2026/09/22 15:47:23 INFO Signed narinfos id=49 count=15838lb # [ 131.219335] niks3-server[1077]: 2026/09/22 15:47:23 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5839worker1 # [ 131.253004] nix-grpc-daemon[1380]: time=2026-09-22T15:47:23.855Z level=INFO msg="Upload complete. (364ms)"5840lb # [ 131.228771] niks3-server[1077]: 2026/09/22 15:47:23 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5841worker1 # [ 131.260156] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=0 paths=1 nar_bytes_in=4165842lb # [ 131.236525] niks3-server[1077]: 2026/09/22 15:47:23 INFO Completed upload id=495843worker1 # [ 131.274777] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358705844worker1 # [ 131.280566] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5845worker1 # [ 131.285461] nix-daemon[829]: reaped child process 1508, status = succeeded5846worker1 # [ 131.295429] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5847client # [ 131.275430] nix[1342]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5848lb: must succeed: curl -sf localhost:9901/clusters5849lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5850lb: must succeed: curl -sf localhost:9901/clusters5851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5852lb: must succeed: curl -sf localhost:9901/clusters5853lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5854lb: must succeed: curl -sf localhost:9901/clusters5855lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5856worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5857worker2 # [ 133.313439] systemd-journald[272]: Received client request to rotate journal, rotating.5858worker2 # [ 133.334677] systemd-journald[272]: Vacuuming done, freed 0B of archived journals from /var/log/journal/59011f543f9f49bda13a45f5715e3f8b.5859worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5860worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5861worker1 # [ 133.415284] systemd[1]: Stopping Nix worker-protocol over gRPC...5862worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5863client: must succeed: sleep 5.05864worker1 # [ 133.654305] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:26Z level=info event=draining5865client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5866lb # [ 138.537897] envoy[894]: conn peer=[2001:db8:1::1]:54122 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=4411 ms=68055867worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5868worker1 # [ 138.733805] nix-daemon[829]: reaped child process 1522, status = succeeded5869worker1 # [ 138.831078] nix-daemon[829]: accepted connection from pid 1563, user nix-grpc-daemon (trusted)5870worker1 # [ 138.851428] nix-daemon[829]: reaped child process 1566, status = succeeded5871lb # [ 138.830514] niks3-server[1077]: 2026/09/22 15:47:31 INFO Received uploads request method=POST path=/api/pending_closures5872worker1 # [ 138.872390] nix-grpc-daemon[1380]: time=2026-09-22T15:47:31.473Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5873worker1 # [ 138.873907] nix-grpc-daemon[1380]: time=2026-09-22T15:47:31.473Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5874lb # [ 138.886574] niks3-server[1077]: 2026/09/22 15:47:31 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5875lb # [ 138.899709] niks3-server[1077]: 2026/09/22 15:47:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5876worker1 # [ 138.930685] nix-grpc-daemon[1380]: time=2026-09-22T15:47:31.533Z level=INFO msg="Uploading 1 narinfos"5877lb # [ 138.910252] niks3-server[1077]: 2026/09/22 15:47:31 INFO Signed narinfos id=50 count=15878lb # [ 138.914621] niks3-server[1077]: 2026/09/22 15:47:31 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5879lb # [ 138.924423] niks3-server[1077]: 2026/09/22 15:47:31 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5880worker1 # [ 138.950611] nix-grpc-daemon[1380]: time=2026-09-22T15:47:31.553Z level=INFO msg="Upload complete. (237ms)"5881lb # [ 138.928247] niks3-server[1077]: 2026/09/22 15:47:31 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5882lb # [ 138.932180] niks3-server[1077]: 2026/09/22 15:47:31 INFO Completed upload id=505883lb # [ 138.935072] niks3-server[1077]: 2026/09/22 15:47:31 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5884worker1 # [ 138.965222] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15885worker1 # [ 138.985935] nix-daemon[829]: reaped child process 1520, status = succeeded5886client # [ 138.980579] systemd[1]: rs.service: Deactivated successfully.5887client # [ 138.985530] systemd[1]: rs.service: Consumed 193ms CPU time over 8.479s wall clock time, 39.5M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.5888worker1 # [ 139.656659] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:32Z level=info event=signal_exit5889lb # [ 139.944669] niks3-server[1077]: 2026/09/22 15:47:32 INFO lead: released remote=192.168.1.3:474865890lb # [ 140.114653] niks3-server[1077]: 2026/09/22 15:47:32 INFO lead: acquired remote=192.168.1.4:459925891worker2 # [ 140.121165] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:32Z level=info event=scheduler_take_over5892worker2 # [ 140.939054] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=scheduler_restarting addr=lb:500515893worker1 # [ 140.969609] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=scheduler_restarting5894worker1 # [ 140.970066] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=scheduler_restarting addr=lb:500515895worker1 # [ 140.970215] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:358625896worker1 # [ 140.970342] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=worker_gone addr=192.168.1.4:500515897worker1 # [ 140.970466] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:358625898worker1 # [ 140.970574] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=worker_gone addr=192.168.1.3:500515899worker1 # [ 140.996975] nix-grpc-daemon[1301]: E0000 00:00:1790092053.599511 1346 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5900worker1 # [ 141.003532] nix-grpc-daemon[1301]: E0000 00:00:1790092053.599767 1346 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5901worker1 # [ 141.033835] nix-daemon[829]: reaped child process 1370, status = succeeded5902worker2 # [ 141.142883] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=scheduler_connected addr=lb:500515903worker2 # [ 141.147159] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5904worker1 # [ 141.188366] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5905worker2 # [ 141.343749] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=scheduler_connected addr=lb:500515906worker2 # [ 141.361527] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=oidc_jwks provider=mock keys=15907worker2 # [ 141.366036] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:403725908worker2 # [ 141.371520] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05909worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.24 seconds)5910client: waiting for success: ! systemctl is-active rs5911client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5912client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5913client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5914client: must fail: journalctl -u rs | grep -q 'reconnecting'5915client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5916lb: must succeed: curl -sf localhost:9901/clusters5917lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5918worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5919worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5920worker1 # [ 142.317965] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5921worker1 # [ 142.324588] systemd[1]: Stopped Nix worker-protocol over gRPC.5922worker1 # [ 142.327192] systemd[1]: nix-grpc-daemon.service: Consumed 1.999s CPU time over 33.842s wall clock time, 38.6M memory peak, 48.3K incoming IP traffic, 91.9K outgoing IP traffic.5923worker1 # [ 142.349125] systemd[1]: Starting Nix worker-protocol over gRPC...5924worker1 # [ 142.502806] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5925worker1 # [ 142.505097] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5926worker1 # [ 142.508919] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15927worker1 # [ 142.514249] nix-grpc-daemon[1573]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5928worker1 # [ 142.515708] nix-grpc-daemon[1573]: E0000 00:00:1790092055.116301 1573 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5929worker1 # [ 142.556267] nix-grpc-daemon[1573]: E0000 00:00:1790092055.122367 1573 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5930worker1 # [ 142.559071] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5931worker1 # [ 142.560974] nix-grpc-daemon[1573]: E0000 00:00:1790092055.127911 1612 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5932worker1 # [ 142.563313] nix-grpc-daemon[1573]: E0000 00:00:1790092055.143706 1612 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5933worker1 # [ 142.568118] nix-grpc-daemon[1573]: E0000 00:00:1790092055.143838 1612 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5934worker1 # [ 142.570485] nix-grpc-daemon[1573]: E0000 00:00:1790092055.158122 1612 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5935worker1 # [ 142.573651] systemd[1]: Started Nix worker-protocol over gRPC.5936worker1 # [ 142.576326] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=scheduler_connected addr=lb:500515937worker2 # [ 142.553151] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:403725938worker2 # [ 142.553431] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05939lb: must succeed: curl -sf localhost:9901/clusters5940lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5941worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5942worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5943(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.93 seconds)5944subtest: scheduler restart mid-build: build finishes, no double build5945client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5946client # [ 143.425088] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5947client # Running as unit: mid.service; invocation ID: 31933518b25a4b6793944f9c357513395948client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.08 seconds)5949client # [ 143.630113] nix[1378]: don't know how to build these paths:5950client # [ 143.631215] nix[1378]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5951worker2 # [ 143.621412] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504805952worker2 # [ 143.635568] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05953worker2 # [ 143.642267] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15954worker2 # [ 143.680834] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504805955worker2 # [ 143.695875] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)5956worker2 # [ 143.787254] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 2/5)5957worker2 # [ 143.871317] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 3/5)5958worker2 # [ 144.043182] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 260 ms (attempt 4/5)5959worker2 # [ 144.306239] nix-grpc-daemon[1482]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5960worker2 # [ 144.312972] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5961worker2 # [ 144.332119] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504805962client # [ 144.352175] nix[1378]: copying 1 paths to 'lb:50051'...5963worker2 # [ 144.347290] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504805964client # [ 144.356399] nix[1378]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5965worker2 # [ 144.365399] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)5966worker2 # [ 144.398576] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=info event=niks3_push_started pid=16105967worker2 # [ 144.547998] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.146Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5968worker2 # [ 144.657342] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)5969worker2 # [ 144.689135] nix-daemon[856]: reaped child process 1626, status = succeeded5970worker2 # [ 144.810369] nix-daemon[856]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)5971worker2 # [ 144.832544] nix-daemon[856]: reaped child process 1635, status = succeeded5972lb # [ 144.831413] niks3-server[1077]: 2026/09/22 15:47:37 INFO Received uploads request method=POST path=/api/pending_closures5973worker2 # [ 144.844094] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.442Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5974worker2 # [ 144.845584] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.442Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5975lb # [ 144.870023] niks3-server[1077]: 2026/09/22 15:47:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5976worker2 # [ 144.883772] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.482Z level=INFO msg="Uploading 1 narinfos"5977lb # [ 144.885927] niks3-server[1077]: 2026/09/22 15:47:37 INFO Signed narinfos id=51 count=15978lb # [ 144.896962] niks3-server[1077]: 2026/09/22 15:47:37 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5979lb # [ 144.918715] niks3-server[1077]: 2026/09/22 15:47:37 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5980worker2 # [ 144.927976] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.527Z level=INFO msg="Upload complete. (375ms)"5981lb # [ 144.935677] niks3-server[1077]: 2026/09/22 15:47:37 INFO Completed upload id=515982worker2 # [ 144.936827] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50480 duration_s=0 paths=1 nar_bytes_in=4245983lb # [ 144.941001] niks3-server[1077]: 2026/09/22 15:47:37 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5984lb # [ 144.944466] niks3-server[1077]: 2026/09/22 15:47:37 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5985worker2 # [ 144.953798] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504805986worker2 # [ 144.960583] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)5987worker2 # [ 144.965950] nix-daemon[856]: reaped child process 1608, status = succeeded5988client # [ 144.983618] nix[1378]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5989worker2 # [ 144.979211] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)5990lb: must succeed: curl -sf localhost:9901/clusters5991lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5992lb: must succeed: curl -sf localhost:9901/clusters5993lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5994worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5995worker2 # [ 147.313962] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1482 (nix-grpc-daemon) on client request.5996worker2 # [ 147.315808] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5997lb # [ 147.323434] niks3-server[1077]: 2026/09/22 15:47:39 INFO lead: released remote=192.168.1.4:459925998client # [ 147.338789] nix[1378]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5999worker2 # [ 147.336667] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6000worker1 # [ 147.357393] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6001worker2 # [ 147.338165] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6002worker2 # [ 147.339405] systemd[1]: nix-grpc-daemon.service: Consumed 600ms CPU time over 17.359s wall clock time, 34.6M memory peak, 15K incoming IP traffic, 25.5K outgoing IP traffic.6003client # [ 147.345024] nix[1378]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6004worker2 # [ 147.356189] systemd[1]: Starting scheduler-token.service...6005worker2 # [ 147.380666] nix-daemon[856]: reaped child process 1637, status = succeeded6006worker2 # [ 147.381640] nix-daemon[856]: reaped child process 1541, status = succeeded6007worker2 # [ 147.388409] nix-daemon[1639]: unexpected Nix daemon error: error: interrupted by the user6008worker2 # [ 147.397337] nix-daemon[856]: reaped child process 1639, status = failed with exit code 16009worker2 # [ 147.421313] systemd[1]: scheduler-token.service: Deactivated successfully.6010worker2 # [ 147.423903] systemd[1]: Finished scheduler-token.service.6011worker2 # [ 147.426904] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 73ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6012worker2 # [ 147.430579] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6013worker2 # [ 147.435630] systemd[1]: Starting scheduler-token.service...6014worker2 # [ 147.477290] systemd[1]: scheduler-token.service: Deactivated successfully.6015worker2 # [ 147.480062] systemd[1]: Finished scheduler-token.service.6016worker2 # [ 147.480869] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 48ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6017worker2 # [ 147.485369] systemd[1]: Starting Nix worker-protocol over gRPC...6018lb # [ 147.502511] niks3-server[1077]: 2026/09/22 15:47:40 INFO lead: acquired remote=192.168.1.3:378386019worker1 # [ 147.533109] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_take_over6020worker1 # [ 147.557571] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:500516021worker2 # [ 147.585342] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6022worker2 # [ 147.587603] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6023worker2 # [ 147.591383] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16024worker2 # [ 147.596050] nix-grpc-daemon[1682]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6025worker2 # [ 147.608135] nix-grpc-daemon[1682]: E0000 00:00:1790092060.195387 1682 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6026worker1 # [ 147.641297] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6027worker2 # [ 147.620067] nix-grpc-daemon[1682]: E0000 00:00:1790092060.198864 1682 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6028worker2 # [ 147.622410] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6029worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.37 seconds)6030client: must succeed: sleep 5.06031worker2 # [ 147.650225] systemd[1]: Started Nix worker-protocol over gRPC.6032worker2 # [ 147.651482] nix-grpc-daemon[1682]: E0000 00:00:1790092060.213695 1721 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6033worker2 # [ 147.654231] nix-grpc-daemon[1682]: E0000 00:00:1790092060.233445 1721 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6034worker2 # [ 147.659264] nix-grpc-daemon[1682]: E0000 00:00:1790092060.233620 1721 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6035worker2 # [ 147.664201] nix-grpc-daemon[1682]: E0000 00:00:1790092060.248838 1721 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6036worker2 # [ 147.667049] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:500516037worker2 # [ 147.668429] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6038worker1 # [ 147.842125] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:500516039worker1 # [ 147.847546] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6040client # [ 147.843256] nix[1378]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting6041worker2 # [ 147.868355] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:500516042worker2 # [ 147.874963] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6043worker1 # [ 148.048407] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:500516044worker1 # [ 148.053691] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6045worker2 # [ 148.076060] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:500516046worker2 # [ 148.082622] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6047worker1 # [ 148.254441] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:500516048worker1 # [ 148.259702] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6049worker2 # [ 148.285480] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:500516050worker2 # [ 148.295717] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6051worker1 # [ 148.462179] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516052worker1 # [ 148.466337] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6053worker2 # [ 148.492825] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516054worker2 # [ 148.498177] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6055lb # [ 148.539017] envoy[894]: conn peer=[2001:db8:1::1]:54124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1517 ms=82576056lb # [ 148.544858] envoy[894]: conn peer=[2001:db8:1::3]:37168 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3358 tx=1078 ms=324756057lb # [ 148.550217] envoy[894]: conn peer=[2001:db8:1::4]:42152 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2679 tx=376 ms=171766058worker1 # [ 148.662724] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516059worker1 # [ 148.667912] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6060worker2 # [ 148.701284] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516061worker2 # [ 148.705758] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6062client # [ 148.847087] nix[1378]: scheduler at lb:50051: no healthy upstream, reconnecting6063worker1 # [ 148.868706] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516064worker1 # [ 148.873846] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6065worker2 # [ 148.902093] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516066worker2 # [ 148.907430] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6067worker1 # [ 149.074756] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516068worker1 # [ 149.079856] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6069worker2 # [ 149.108855] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516070worker2 # [ 149.114473] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6071worker1 # [ 149.280719] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516072worker1 # [ 149.285842] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6073worker2 # [ 149.315241] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:500516074worker2 # [ 149.321771] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6075worker1 # [ 149.487817] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=scheduler_connected addr=lb:500516076worker1 # [ 149.505196] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:581806077worker1 # [ 149.510290] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06078worker2 # [ 149.522445] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:42Z level=info event=scheduler_connected addr=lb:500516079worker1 # [ 149.557678] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=oidc_jwks provider=mock keys=16080worker1 # [ 149.562131] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:581806081worker1 # [ 149.567504] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06082worker1 # [ 150.878547] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:43Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06083worker1 # [ 150.878854] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:43Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16084worker1 # [ 150.937815] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422146085worker1 # [ 150.954058] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6086worker1 # [ 150.976164] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)6087worker1 # [ 151.074148] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5)6088worker1 # [ 151.252329] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 3/5)6089worker1 # [ 151.314996] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 624 ms (attempt 4/5)6090worker1 # [ 151.942503] nix-grpc-daemon[1573]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6091worker1 # [ 151.949761] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6092worker1 # [ 151.964909] nix-grpc-daemon[1573]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...6093worker1 # [ 151.989512] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6094client # [ 151.992925] nix[1378]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6095worker1 # [ 152.015255] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6096client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6097client: waiting for success: ! systemctl is-active mid6098worker1 # [ 243.305805] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:15Z level=info event=niks3_push_started pid=16866099worker1 # [ 243.368700] nix-daemon[829]: reaped child process 1656, status = succeeded6100worker1 # [ 243.497080] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.099Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6101worker1 # [ 243.604100] nix-daemon[829]: accepted connection from pid 1698, user nix-grpc-daemon (trusted)6102worker1 # [ 243.625337] nix-daemon[829]: reaped child process 1701, status = succeeded6103lb # [ 243.599817] niks3-server[1077]: 2026/09/22 15:49:16 INFO Received uploads request method=POST path=/api/pending_closures6104worker1 # [ 243.640066] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.241Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6105worker1 # [ 243.641548] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.241Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6106lb # [ 243.661951] niks3-server[1077]: 2026/09/22 15:49:16 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6107lb # [ 243.679017] niks3-server[1077]: 2026/09/22 15:49:16 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6108worker1 # [ 243.714852] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.316Z level=INFO msg="Uploading 1 narinfos"6109lb # [ 243.688542] niks3-server[1077]: 2026/09/22 15:49:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6110lb # [ 243.691751] niks3-server[1077]: 2026/09/22 15:49:16 INFO Signed narinfos id=52 count=16111lb # [ 243.701677] niks3-server[1077]: 2026/09/22 15:49:16 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6112lb # [ 243.708045] niks3-server[1077]: 2026/09/22 15:49:16 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6113lb # [ 243.717977] niks3-server[1077]: 2026/09/22 15:49:16 INFO Completed upload id=526114worker1 # [ 243.747121] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.349Z level=INFO msg="Upload complete. (246ms)"6115lb # [ 243.723622] niks3-server[1077]: 2026/09/22 15:49:16 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6116worker1 # [ 243.764454] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42214 duration_s=92 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16117worker1 # [ 243.782862] nix-daemon[829]: reaped child process 1654, status = succeeded6118client # [ 243.777764] systemd[1]: mid.service: Deactivated successfully.6119client # [ 243.783149] systemd[1]: mid.service: Consumed 255ms CPU time over 1min 40.355s wall clock time, 41.4M memory peak, 4K written to disk, 9.2K incoming IP traffic, 11.2K outgoing IP traffic.6120client: (finished: waiting for success: ! systemctl is-active mid, in 91.01 seconds)6121client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6122client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)6123(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 100.70 seconds)6124subtest: interrupting the client stops the build on the worker6125client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6126client # [ 244.092506] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6127client # Running as unit: intr.service; invocation ID: ff987b96ef5e46c093253097b9f166a86128client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6129worker2 # [ 244.283377] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377106130worker2 # [ 244.297879] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6131client # [ 244.321717] nix[1839]: don't know how to build these paths:6132client # [ 244.324979] nix[1839]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6133worker1 # [ 244.357011] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:16Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06134worker1 # [ 244.363082] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:16Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16135worker2 # [ 244.378458] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106136worker2 # [ 244.395100] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 1/5)6137worker2 # [ 244.431259] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)6138worker2 # [ 244.548388] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 249 ms (attempt 3/5)6139worker2 # [ 244.799322] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 4/5)6140worker2 # [ 245.075504] nix-grpc-daemon[1682]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6141worker2 # [ 245.082251] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6142worker2 # [ 245.100772] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377106143client # [ 245.123208] nix[1839]: copying 1 paths to 'lb:50051'...6144worker2 # [ 245.116133] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377106145client # [ 245.123662] nix[1839]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6146worker2 # [ 245.130842] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6147worker2 # [ 245.157703] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:17Z level=info event=niks3_push_started pid=17546148worker2 # [ 245.322550] nix-grpc-daemon[1754]: time=2026-09-22T15:49:17.921Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6149worker2 # [ 245.453697] nix-daemon[856]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6150worker2 # [ 245.473219] nix-daemon[856]: reaped child process 1774, status = succeeded6151worker2 # [ 245.578195] nix-daemon[856]: accepted connection from pid 1776, user nix-grpc-daemon (trusted)6152worker2 # [ 245.597474] nix-daemon[856]: reaped child process 1779, status = succeeded6153lb # [ 245.595553] niks3-server[1077]: 2026/09/22 15:49:18 INFO Received uploads request method=POST path=/api/pending_closures6154worker2 # [ 245.607725] nix-grpc-daemon[1754]: time=2026-09-22T15:49:18.205Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6155worker2 # [ 245.609849] nix-grpc-daemon[1754]: time=2026-09-22T15:49:18.205Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6156lb # [ 245.631169] niks3-server[1077]: 2026/09/22 15:49:18 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6157lb # [ 245.645808] niks3-server[1077]: 2026/09/22 15:49:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6158worker2 # [ 245.653665] nix-grpc-daemon[1754]: time=2026-09-22T15:49:18.252Z level=INFO msg="Uploading 1 narinfos"6159lb # [ 245.660381] niks3-server[1077]: 2026/09/22 15:49:18 INFO Signed narinfos id=53 count=16160lb # [ 245.663456] niks3-server[1077]: 2026/09/22 15:49:18 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6161lb # [ 245.680199] niks3-server[1077]: 2026/09/22 15:49:18 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6162lb # [ 245.687946] niks3-server[1077]: 2026/09/22 15:49:18 INFO Completed upload id=536163worker2 # [ 245.688392] nix-grpc-daemon[1754]: time=2026-09-22T15:49:18.287Z level=INFO msg="Upload complete. (361ms)"6164lb # [ 245.690830] niks3-server[1077]: 2026/09/22 15:49:18 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6165worker2 # [ 245.696947] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=4246166worker2 # [ 245.711546] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106167worker2 # [ 245.719174] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6168worker2 # [ 245.724100] nix-daemon[856]: reaped child process 1752, status = succeeded6169client # [ 245.742172] nix[1839]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6170worker2 # [ 245.737924] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6171client: must succeed: systemctl kill -s INT intr6172client # [ 247.790579] systemd[1]: intr.service: Sent signal SIGINT to main process 1839 (nix) on client request.6173worker2 # [ 247.787064] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:20Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6174client # [ 247.797112] nix[1839]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6175client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6176client # [ 247.804931] nix[1839]: [mutex.cc : 451] RAW: Unlock 0xed2e75dfc788 @ 0xed327a4359a4 0xed327b50d40c 0xed329a9a57bc 0xed329a9a7414 0xed329a99e1d4 0xed32999e4f7c 0xed3299772d806177client # [ 247.821703] systemd-coredump[1853]: Process 1839 (nix) of user 0 terminated abnormally with signal 11/SEGV, processing...6178client # [ 247.843314] systemd[1]: Created slice Slice /system/systemd-coredump.6179client # [ 247.851756] systemd[1]: Started Process Core Dump (PID 1853/UID 0).6180worker2 # [ 247.963827] nix-daemon[1783]: unexpected Nix daemon error: error: interrupted by the user6181worker2 # [ 247.969454] nix-daemon[856]: reaped child process 1781, status = succeeded6182worker2 # [ 247.974255] nix-daemon[856]: reaped child process 1783, status = failed with exit code 16183lb # [ 248.549867] envoy[894]: conn peer=[2001:db8:1::1]:35062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4288 tx=2113 ms=1001296184(finished: subtest: interrupting the client stops the build on the worker, in 5.01 seconds)6185subtest: a deploy mid-build drains: the build finishes, the new generation takes over6186client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6187client # [ 249.119178] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6188client # Running as unit: sw.service; invocation ID: fbb71412e3ba4a83846065debe3274566189client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.06 seconds)6190client # [ 249.504909] nix[1860]: don't know how to build these paths:6191worker1 # [ 249.523187] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422146192client # [ 249.509849] nix[1860]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6193worker1 # [ 249.540160] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06194worker1 # [ 249.546792] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16195worker1 # [ 249.596398] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422146196worker1 # [ 249.615775] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422146197client # [ 249.613216] nix[1860]: copying 1 paths to 'lb:50051'...6198worker1 # [ 249.630619] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422146199client # [ 249.617621] nix[1860]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6200worker1 # [ 249.649161] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6201worker1 # [ 249.812460] nix-daemon[829]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6202worker1 # [ 249.842845] nix-daemon[829]: reaped child process 1742, status = succeeded6203worker1 # [ 249.935864] nix-daemon[829]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6204worker1 # [ 249.955057] nix-daemon[829]: reaped child process 1747, status = succeeded6205lb # [ 249.932836] niks3-server[1077]: 2026/09/22 15:49:22 INFO Received uploads request method=POST path=/api/pending_closures6206worker1 # [ 249.970050] nix-grpc-daemon[1686]: time=2026-09-22T15:49:22.572Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6207worker1 # [ 249.971515] nix-grpc-daemon[1686]: time=2026-09-22T15:49:22.572Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6208lb # [ 249.961291] niks3-server[1077]: 2026/09/22 15:49:22 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6209lb # [ 249.976510] niks3-server[1077]: 2026/09/22 15:49:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6210worker1 # [ 250.007872] nix-grpc-daemon[1686]: time=2026-09-22T15:49:22.610Z level=INFO msg="Uploading 1 narinfos"6211lb # [ 249.985700] niks3-server[1077]: 2026/09/22 15:49:22 INFO Signed narinfos id=54 count=16212lb # [ 249.991042] niks3-server[1077]: 2026/09/22 15:49:22 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6213lb # [ 250.000328] niks3-server[1077]: 2026/09/22 15:49:22 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6214worker1 # [ 250.032220] nix-grpc-daemon[1686]: time=2026-09-22T15:49:22.635Z level=INFO msg="Upload complete. (365ms)"6215lb # [ 250.008403] niks3-server[1077]: 2026/09/22 15:49:22 INFO Completed upload id=546216lb # [ 250.010460] niks3-server[1077]: 2026/09/22 15:49:22 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6217worker1 # [ 250.040349] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42214 duration_s=0 paths=1 nar_bytes_in=4166218worker1 # [ 250.052343] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422146219worker1 # [ 250.061442] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6220worker1 # [ 250.065911] nix-daemon[829]: reaped child process 1737, status = succeeded6221client # [ 250.056375] nix[1860]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6222worker1 # [ 250.078270] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6223client # [ 250.386161] systemd-coredump[1854]: Cannot store coredump of 1839 (nix): No space left on device6224client # [ 250.523450] systemd-coredump[1854]: Process 1839 (nix) of user 0 terminated abnormally without generating a coredump.6225client # [ 250.540785] systemd[1]: systemd-coredump@0-1-1853_1854-0.service: Deactivated successfully.6226client # [ 250.542337] systemd[1]: systemd-coredump@0-1-1853_1854-0.service: Consumed 2.018s CPU time over 2.683s wall clock time, 968.7M memory peak, 926.2M written to disk.6227client # [ 250.550494] systemd[1]: intr.service: Main process exited, code=dumped, status=11/SEGV6228client # [ 250.551915] systemd[1]: intr.service: Failed with result 'core-dump'.6229client # [ 250.554195] systemd[1]: intr.service: Consumed 365ms CPU time over 6.458s wall clock time, 41.5M memory peak, 4K written to disk, 6.5K incoming IP traffic, 8.2K outgoing IP traffic.6230worker1: must succeed: systemctl reload nix-grpc-daemon.service6231worker1 # [ 251.730007] systemd[1]: Reloading Nix worker-protocol over gRPC...6232worker1 # [ 251.775965] systemd[1]: Reloaded Nix worker-protocol over gRPC.6233worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6234worker1: must succeed: pgrep -f 'read -t [9]0 x'6235worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6236client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26237worker2 # [ 252.021947] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377106238client # don't know how to build these paths:6239client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6240worker2 # [ 252.033362] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377106241worker1 # [ 252.082109] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:24Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06242worker1 # [ 252.084088] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:24Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16243worker2 # [ 252.100514] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106244worker2 # [ 252.115437] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377106245client # copying 1 paths to 'lb:50051'...6246client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6247worker2 # [ 252.130852] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377106248worker2 # [ 252.145712] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6249worker2 # [ 252.303866] nix-daemon[856]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6250worker2 # [ 252.322367] nix-daemon[856]: reaped child process 1837, status = succeeded6251worker2 # [ 252.425741] nix-daemon[856]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6252worker2 # [ 252.445020] nix-daemon[856]: reaped child process 1842, status = succeeded6253lb # [ 252.450431] niks3-server[1077]: 2026/09/22 15:49:25 INFO Received uploads request method=POST path=/api/pending_closures6254worker2 # [ 252.460447] nix-grpc-daemon[1754]: time=2026-09-22T15:49:25.059Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6255worker2 # [ 252.462396] nix-grpc-daemon[1754]: time=2026-09-22T15:49:25.061Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6256lb # [ 252.480988] niks3-server[1077]: 2026/09/22 15:49:25 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6257worker2 # [ 252.493525] nix-grpc-daemon[1754]: time=2026-09-22T15:49:25.092Z level=INFO msg="Uploading 1 narinfos"6258lb # [ 252.493481] niks3-server[1077]: 2026/09/22 15:49:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6259lb # [ 252.506555] niks3-server[1077]: 2026/09/22 15:49:25 INFO Signed narinfos id=55 count=16260worker2 # [ 252.515573] nix-grpc-daemon[1754]: time=2026-09-22T15:49:25.114Z level=INFO msg="Upload complete. (352ms)"6261lb # [ 252.514770] niks3-server[1077]: 2026/09/22 15:49:25 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6262lb # [ 252.521596] niks3-server[1077]: 2026/09/22 15:49:25 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6263worker2 # [ 252.524256] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=5286264lb # [ 252.526834] niks3-server[1077]: 2026/09/22 15:49:25 INFO Completed upload id=556265worker1 # [ 252.555824] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:25Z level=info event=draining6266lb # [ 252.529309] niks3-server[1077]: 2026/09/22 15:49:25 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6267worker2 # [ 252.535892] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106268worker2 # [ 252.541922] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6269worker2 # [ 252.547219] nix-daemon[856]: reaped child process 1832, status = succeeded6270worker2 # [ 252.557043] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6271client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6272client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6273client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6274client # farm-a> LOG-farm-a-during-drain6275worker2 # [ 253.655585] nix-daemon[856]: reaped child process 1846, status = succeeded6276worker2 # [ 253.734339] nix-daemon[856]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6277worker2 # [ 253.753558] nix-daemon[856]: reaped child process 1865, status = succeeded6278lb # [ 253.758619] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received uploads request method=POST path=/api/pending_closures6279worker2 # [ 253.771628] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.367Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6280worker2 # [ 253.774649] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.367Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6281lb # [ 253.791707] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6282lb # [ 253.809458] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6283lb # [ 253.826004] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6284worker2 # [ 253.827814] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.426Z level=INFO msg="Uploading 1 narinfos"6285lb # [ 253.833410] niks3-server[1077]: 2026/09/22 15:49:26 INFO Signed narinfos id=56 count=16286lb # [ 253.838751] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6287lb # [ 253.849241] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6288lb # [ 253.856302] niks3-server[1077]: 2026/09/22 15:49:26 INFO Completed upload id=566289worker2 # [ 253.856388] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.455Z level=INFO msg="Upload complete. (288ms)"6290lb # [ 253.859068] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6291worker1 # [ 253.896913] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:26Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06292worker1 # [ 253.898824] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:26Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16293worker2 # [ 253.869140] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16294worker2 # [ 253.889435] nix-daemon[856]: reaped child process 1844, status = succeeded6295worker2 # [ 253.921810] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106296worker2 # [ 253.936903] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377106297worker2 # [ 253.951323] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.550Z level=INFO msg="All 1 paths already cached"6298client # copying 1 paths to 'lb:50051'...6299client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6300worker2 # [ 253.962185] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377106301worker2 # [ 253.979594] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6302worker2 # [ 254.130258] nix-daemon[856]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)6303worker2 # [ 254.149128] nix-daemon[856]: reaped child process 1872, status = succeeded6304worker2 # [ 254.249859] nix-daemon[856]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6305worker2 # [ 254.268408] nix-daemon[856]: reaped child process 1877, status = succeeded6306lb # [ 254.266411] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received uploads request method=POST path=/api/pending_closures6307worker2 # [ 254.276155] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.875Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6308worker2 # [ 254.278229] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.877Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6309lb # [ 254.295826] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6310lb # [ 254.311065] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6311worker2 # [ 254.318659] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.917Z level=INFO msg="Uploading 1 narinfos"6312lb # [ 254.323950] niks3-server[1077]: 2026/09/22 15:49:26 INFO Signed narinfos id=57 count=16313lb # [ 254.329723] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6314worker2 # [ 254.340104] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.939Z level=INFO msg="Upload complete. (348ms)"6315lb # [ 254.340775] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6316lb # [ 254.346870] niks3-server[1077]: 2026/09/22 15:49:26 INFO Completed upload id=576317worker2 # [ 254.348122] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=6406318lb # [ 254.349925] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6319worker2 # [ 254.358643] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106320worker2 # [ 254.366666] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6321worker2 # [ 254.368900] nix-daemon[856]: reaped child process 1867, status = succeeded6322client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6323worker2 # [ 254.381218] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6324client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6325client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6326client # farm-b> LOG-farm-b-during-drain6327worker2 # [ 255.848910] nix-daemon[856]: reaped child process 1881, status = succeeded6328worker2 # [ 255.956819] nix-daemon[856]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6329worker2 # [ 255.975220] nix-daemon[856]: reaped child process 1900, status = succeeded6330lb # [ 255.978974] niks3-server[1077]: 2026/09/22 15:49:28 INFO Received uploads request method=POST path=/api/pending_closures6331worker2 # [ 255.989669] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.588Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6332worker2 # [ 255.991442] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.588Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6333lb # [ 256.015851] niks3-server[1077]: 2026/09/22 15:49:28 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6334worker2 # [ 256.040917] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.639Z level=INFO msg="Uploading 1 narinfos"6335lb # [ 256.028920] niks3-server[1077]: 2026/09/22 15:49:28 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6336lb # [ 256.044524] niks3-server[1077]: 2026/09/22 15:49:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6337lb # [ 256.047774] niks3-server[1077]: 2026/09/22 15:49:28 INFO Signed narinfos id=58 count=16338lb # [ 256.064172] niks3-server[1077]: 2026/09/22 15:49:28 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6339lb # [ 256.071240] niks3-server[1077]: 2026/09/22 15:49:28 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6340lb # [ 256.075047] niks3-server[1077]: 2026/09/22 15:49:28 INFO Completed upload id=586341lb # [ 256.077074] niks3-server[1077]: 2026/09/22 15:49:28 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6342worker2 # [ 256.075920] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.675Z level=INFO msg="Upload complete. (284ms)"6343worker1 # [ 256.114021] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:28Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06344worker1 # [ 256.115951] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:28Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16345worker2 # [ 256.090353] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16346worker2 # [ 256.108697] nix-daemon[856]: reaped child process 1879, status = succeeded6347worker2 # [ 256.132878] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106348worker2 # [ 256.148788] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377106349worker2 # [ 256.163515] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.762Z level=INFO msg="All 2 paths already cached"6350client # copying 1 paths to 'lb:50051'...6351client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6352worker2 # [ 256.174784] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377106353worker2 # [ 256.188885] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6354worker2 # [ 256.338166] nix-daemon[856]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)6355worker2 # [ 256.356132] nix-daemon[856]: reaped child process 1907, status = succeeded6356worker2 # [ 256.456153] nix-daemon[856]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)6357worker2 # [ 256.472697] nix-daemon[856]: reaped child process 1912, status = succeeded6358lb # [ 256.477472] niks3-server[1077]: 2026/09/22 15:49:29 INFO Received uploads request method=POST path=/api/pending_closures6359worker2 # [ 256.486825] nix-grpc-daemon[1754]: time=2026-09-22T15:49:29.085Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6360worker2 # [ 256.488504] nix-grpc-daemon[1754]: time=2026-09-22T15:49:29.085Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6361lb # [ 256.506860] niks3-server[1077]: 2026/09/22 15:49:29 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6362worker2 # [ 256.520425] nix-grpc-daemon[1754]: time=2026-09-22T15:49:29.119Z level=INFO msg="Uploading 1 narinfos"6363lb # [ 256.523228] niks3-server[1077]: 2026/09/22 15:49:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6364lb # [ 256.533895] niks3-server[1077]: 2026/09/22 15:49:29 INFO Signed narinfos id=59 count=16365lb # [ 256.538664] niks3-server[1077]: 2026/09/22 15:49:29 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6366lb # [ 256.547842] niks3-server[1077]: 2026/09/22 15:49:29 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6367lb # [ 256.554430] niks3-server[1077]: 2026/09/22 15:49:29 INFO Completed upload id=596368worker2 # [ 256.554791] nix-grpc-daemon[1754]: time=2026-09-22T15:49:29.154Z level=INFO msg="Upload complete. (350ms)"6369lb # [ 256.557708] niks3-server[1077]: 2026/09/22 15:49:29 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6370worker2 # [ 256.564308] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=7686371worker2 # [ 256.575279] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106372worker2 # [ 256.584574] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6373worker2 # [ 256.587475] nix-daemon[856]: reaped child process 1902, status = succeeded6374client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6375worker2 # [ 256.598693] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6376client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6377client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6378client # farm-top> LOG-farm-top-during-drain6379worker2 # [ 258.068869] nix-daemon[856]: reaped child process 1916, status = succeeded6380worker2 # [ 258.172788] nix-daemon[856]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6381worker2 # [ 258.191050] nix-daemon[856]: reaped child process 1935, status = succeeded6382lb # [ 258.197785] niks3-server[1077]: 2026/09/22 15:49:30 INFO Received uploads request method=POST path=/api/pending_closures6383worker2 # [ 258.209877] nix-grpc-daemon[1754]: time=2026-09-22T15:49:30.807Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6384worker2 # [ 258.211717] nix-grpc-daemon[1754]: time=2026-09-22T15:49:30.807Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6385lb # [ 258.236613] niks3-server[1077]: 2026/09/22 15:49:30 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6386lb # [ 258.250173] niks3-server[1077]: 2026/09/22 15:49:30 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6387lb # [ 258.262289] niks3-server[1077]: 2026/09/22 15:49:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6388worker2 # [ 258.268462] nix-grpc-daemon[1754]: time=2026-09-22T15:49:30.867Z level=INFO msg="Uploading 1 narinfos"6389lb # [ 258.272852] niks3-server[1077]: 2026/09/22 15:49:30 INFO Signed narinfos id=60 count=16390lb # [ 258.279925] niks3-server[1077]: 2026/09/22 15:49:30 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6391worker2 # [ 258.293415] nix-grpc-daemon[1754]: time=2026-09-22T15:49:30.892Z level=INFO msg="Upload complete. (281ms)"6392lb # [ 258.292979] niks3-server[1077]: 2026/09/22 15:49:30 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6393lb # [ 258.297552] niks3-server[1077]: 2026/09/22 15:49:30 INFO Completed upload id=606394lb # [ 258.301009] niks3-server[1077]: 2026/09/22 15:49:30 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6395worker2 # [ 258.305196] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16396worker2 # [ 258.325169] nix-daemon[856]: reaped child process 1914, status = succeeded6397client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.55 seconds)6398client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6399client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.17 seconds)6400worker1: must succeed: pkill -f 'read -t [9]0 x'6401lb # [ 258.552288] envoy[894]: conn peer=[2001:db8:1::1]:43232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554536:system_library:OPENSSL_internal:Connection_reset_by_peer:TLS_error_end" rx=3000 tx=913 ms=62696402lb # [ 258.560644] envoy[894]: conn peer=[2001:db8:1::1]:57646 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9638 tx=4451 ms=63006403worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6404client: waiting for success: ! systemctl is-active sw6405worker1 # [ 258.724829] nix-daemon[829]: reaped child process 1751, status = succeeded6406worker1 # [ 258.851432] nix-daemon[829]: accepted connection from pid 1798, user nix-grpc-daemon (trusted)6407worker1 # [ 258.869013] nix-daemon[829]: reaped child process 1801, status = succeeded6408lb # [ 258.848256] niks3-server[1077]: 2026/09/22 15:49:31 INFO Received uploads request method=POST path=/api/pending_closures6409worker1 # [ 258.883908] nix-grpc-daemon[1686]: time=2026-09-22T15:49:31.485Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6410worker1 # [ 258.885668] nix-grpc-daemon[1686]: time=2026-09-22T15:49:31.485Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6411lb # [ 258.877512] niks3-server[1077]: 2026/09/22 15:49:31 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6412lb # [ 258.893725] niks3-server[1077]: 2026/09/22 15:49:31 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6413lb # [ 258.910393] niks3-server[1077]: 2026/09/22 15:49:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6414worker1 # [ 258.938560] nix-grpc-daemon[1686]: time=2026-09-22T15:49:31.539Z level=INFO msg="Uploading 1 narinfos"6415lb # [ 258.917921] niks3-server[1077]: 2026/09/22 15:49:31 INFO Signed narinfos id=61 count=16416lb # [ 258.926441] niks3-server[1077]: 2026/09/22 15:49:31 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6417worker1 # [ 258.967178] nix-grpc-daemon[1686]: time=2026-09-22T15:49:31.569Z level=INFO msg="Upload complete. (272ms)"6418lb # [ 258.942405] niks3-server[1077]: 2026/09/22 15:49:31 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6419lb # [ 258.946513] niks3-server[1077]: 2026/09/22 15:49:31 INFO Completed upload id=616420lb # [ 258.951448] niks3-server[1077]: 2026/09/22 15:49:31 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6421worker1 # [ 258.981155] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42214 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16422worker1 # [ 259.001278] nix-daemon[829]: reaped child process 1749, status = succeeded6423client # [ 259.008747] systemd[1]: sw.service: Deactivated successfully.6424client # [ 259.013662] systemd[1]: sw.service: Consumed 199ms CPU time over 9.890s wall clock time, 41.4M memory peak, 4K written to disk, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6425worker1 # [ 259.560851] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:32Z level=info event=signal_exit6426client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6427client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6428client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6429worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6430lb # [ 260.513910] niks3-server[1077]: 2026/09/22 15:49:33 INFO lead: released remote=192.168.1.3:378386431worker1 # [ 261.533614] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=scheduler_restarting6432worker2 # [ 261.509663] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_restarting addr=lb:500516433worker1 # [ 261.546081] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:581806434worker1 # [ 261.550915] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=worker_gone addr=192.168.1.4:500516435worker1 # [ 261.555602] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=scheduler_restarting addr=lb:500516436worker1 # [ 261.561227] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:581806437worker1 # [ 261.565901] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=worker_gone addr=192.168.1.3:500516438worker1 # [ 261.588618] nix-daemon[829]: reaped child process 1647, status = succeeded6439worker1 # [ 261.592506] nix-grpc-daemon[1573]: E0000 00:00:1790092174.194884 1618 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6440worker1 # [ 261.598867] nix-grpc-daemon[1573]: E0000 00:00:1790092174.195284 1618 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6441worker2 # [ 261.710122] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_connected addr=lb:500516442worker2 # [ 261.715648] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6443worker1 # [ 261.760561] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6444worker2 # [ 261.916659] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_connected addr=lb:500516445worker2 # [ 261.922071] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6446worker2 # [ 262.123239] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_connected addr=lb:500516447worker2 # [ 262.129113] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6448worker2 # [ 262.330328] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_connected addr=lb:500516449worker2 # [ 262.336700] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6450worker2 # [ 262.539648] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:500516451worker2 # [ 262.544337] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6452worker2 # [ 262.609669] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_take_over6453lb # [ 262.609655] niks3-server[1077]: 2026/09/22 15:49:35 INFO lead: acquired remote=192.168.1.4:451506454worker1 # [ 262.658291] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6455worker1 # [ 262.664225] systemd[1]: nix-grpc-daemon.service: Consumed 1.145s CPU time over 2min 310ms wall clock time, 36.8M memory peak, 27.7K incoming IP traffic, 36.5K outgoing IP traffic.6456worker1 # [ 262.687189] systemd[1]: Starting Nix worker-protocol over gRPC...6457worker2 # [ 262.741054] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:500516458worker2 # [ 262.747061] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6459worker1 # [ 262.822550] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6460worker1 # [ 262.824936] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6461worker1 # [ 262.828752] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16462worker1 # [ 262.835388] nix-grpc-daemon[1815]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6463worker1 # [ 262.856215] nix-grpc-daemon[1815]: E0000 00:00:1790092175.435967 1815 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6464worker1 # [ 262.858565] nix-grpc-daemon[1815]: E0000 00:00:1790092175.438135 1815 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6465worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.13 seconds)6466lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6467lb: must succeed: curl -sf localhost:9901/clusters6468worker1 # [ 262.884651] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6469worker1 # [ 262.886899] nix-grpc-daemon[1815]: E0000 00:00:1790092175.447841 1854 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6470worker1 # [ 262.889603] nix-grpc-daemon[1815]: E0000 00:00:1790092175.469506 1854 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6471worker1 # [ 262.891819] nix-grpc-daemon[1815]: E0000 00:00:1790092175.469725 1854 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6472worker1 # [ 262.894376] nix-grpc-daemon[1815]: E0000 00:00:1790092175.486555 1854 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6473worker1 # [ 262.899850] systemd[1]: Started Nix worker-protocol over gRPC.6474worker1 # [ 262.902774] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:500516475worker1 # [ 262.904139] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6476lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6477worker2 # [ 262.948321] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:500516478worker2 # [ 262.953655] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6479worker1 # [ 263.103910] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:500516480worker1 # [ 263.109106] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6481worker2 # [ 263.155154] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:500516482worker2 # [ 263.160627] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6483worker1 # [ 263.309862] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:500516484worker1 # [ 263.315208] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6485worker2 # [ 263.361767] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:500516486worker2 # [ 263.367251] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6487worker1 # [ 263.516180] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:36Z level=info event=scheduler_connected addr=lb:500516488worker1 # [ 263.521620] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6489worker2 # [ 263.570742] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=scheduler_connected addr=lb:500516490worker2 # [ 263.575157] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6491worker1 # [ 263.722257] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:36Z level=info event=scheduler_connected addr=lb:500516492worker2 # [ 263.713782] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:474706493worker2 # [ 263.719564] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06494worker2 # [ 263.771031] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=scheduler_connected addr=lb:500516495worker2 # [ 263.784137] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=oidc_jwks provider=mock keys=16496worker2 # [ 263.790087] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:474706497worker2 # [ 263.795649] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06498lb: must succeed: curl -sf localhost:9901/clusters6499lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6500lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)6501lb: must succeed: curl -sf localhost:9901/clusters6502lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6503worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6504worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6505(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.06 seconds)6506subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6507client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16508worker2 # [ 264.461136] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377106509worker2 # [ 264.469588] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377106510worker2 # [ 264.514323] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06511worker2 # [ 264.520807] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16512worker2 # [ 264.526939] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06513worker2 # [ 264.532364] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06514worker1 # [ 264.598385] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686515worker1 # [ 264.614116] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6516worker1 # [ 264.639248] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)6517worker1 # [ 264.709253] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 2/5)6518worker1 # [ 264.768089] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 3/5)6519worker1 # [ 264.966364] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 512 ms (attempt 4/5)6520worker1 # [ 265.480574] nix-grpc-daemon[1815]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6521worker1 # [ 265.486666] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6522worker1 # [ 265.503382] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686523worker1 # [ 265.518255] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686524worker1 # [ 265.532230] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6525worker1 # [ 265.558351] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=info event=niks3_push_started pid=18756526worker1 # [ 265.706837] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.309Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6527worker1 # [ 265.815409] nix-daemon[829]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6528worker1 # [ 265.832252] nix-daemon[829]: reaped child process 1891, status = succeeded6529worker1 # [ 265.935088] nix-daemon[829]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)6530worker1 # [ 265.950748] nix-daemon[829]: reaped child process 1896, status = succeeded6531lb # [ 265.931760] niks3-server[1077]: 2026/09/22 15:49:38 INFO Received uploads request method=POST path=/api/pending_closures6532worker1 # [ 265.968064] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.570Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6533worker1 # [ 265.969539] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.570Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6534lb # [ 265.970006] niks3-server[1077]: 2026/09/22 15:49:38 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6535worker1 # [ 266.016758] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.619Z level=INFO msg="Uploading 1 narinfos"6536lb # [ 265.989798] niks3-server[1077]: 2026/09/22 15:49:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6537lb # [ 266.000776] niks3-server[1077]: 2026/09/22 15:49:38 INFO Signed narinfos id=62 count=16538lb # [ 266.008675] niks3-server[1077]: 2026/09/22 15:49:38 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6539lb # [ 266.029699] niks3-server[1077]: 2026/09/22 15:49:38 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6540worker1 # [ 266.063108] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.665Z level=INFO msg="Upload complete. (352ms)"6541lb # [ 266.040592] niks3-server[1077]: 2026/09/22 15:49:38 INFO Completed upload id=626542worker1 # [ 266.075273] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=5126543worker1 # [ 266.097495] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686544worker1 # [ 266.104840] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6545worker1 # [ 266.109383] nix-daemon[829]: reaped child process 1873, status = succeeded6546worker1 # [ 266.122436] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6547worker1 # [ 266.958120] nix-daemon[829]: reaped child process 1900, status = succeeded6548worker1 # [ 267.075552] nix-daemon[829]: accepted connection from pid 1916, user nix-grpc-daemon (trusted)6549worker1 # [ 267.093928] nix-daemon[829]: reaped child process 1919, status = succeeded6550lb # [ 267.075095] niks3-server[1077]: 2026/09/22 15:49:39 INFO Received uploads request method=POST path=/api/pending_closures6551worker1 # [ 267.120447] nix-grpc-daemon[1875]: time=2026-09-22T15:49:39.720Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6552worker1 # [ 267.122238] nix-grpc-daemon[1875]: time=2026-09-22T15:49:39.720Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6553worker1 # [ 267.170632] nix-grpc-daemon[1875]: time=2026-09-22T15:49:39.772Z level=INFO msg="Uploading 1 narinfos"6554lb # [ 267.132911] niks3-server[1077]: 2026/09/22 15:49:39 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6555lb # [ 267.149516] niks3-server[1077]: 2026/09/22 15:49:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6556lb # [ 267.163485] niks3-server[1077]: 2026/09/22 15:49:39 INFO Signed narinfos id=63 count=16557lb # [ 267.170030] niks3-server[1077]: 2026/09/22 15:49:39 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6558lb # [ 267.181087] niks3-server[1077]: 2026/09/22 15:49:39 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6559worker1 # [ 267.212710] nix-grpc-daemon[1875]: time=2026-09-22T15:49:39.815Z level=INFO msg="Upload complete. (280ms)"6560lb # [ 267.187649] niks3-server[1077]: 2026/09/22 15:49:39 INFO Completed upload id=636561lb # [ 267.195670] niks3-server[1077]: 2026/09/22 15:49:39 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6562lb # [ 267.203105] niks3-server[1077]: 2026/09/22 15:49:39 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6563worker2 # [ 267.201059] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:39Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16564worker1 # [ 267.225232] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16565worker2 # [ 267.216280] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:39Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06566worker1 # [ 267.246785] nix-daemon[829]: reaped child process 1898, status = succeeded6567worker1 # [ 267.273690] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686568worker1 # [ 267.289339] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686569worker1 # [ 267.303965] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686570worker1 # [ 267.317832] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6571worker1 # [ 267.474561] nix-daemon[829]: accepted connection from pid 1923, user nix-grpc-daemon (trusted)6572worker1 # [ 267.502226] nix-daemon[829]: reaped child process 1926, status = succeeded6573worker1 # [ 267.594501] nix-daemon[829]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)6574worker1 # [ 267.613544] nix-daemon[829]: reaped child process 1931, status = succeeded6575lb # [ 267.584993] niks3-server[1077]: 2026/09/22 15:49:40 INFO Received uploads request method=POST path=/api/pending_closures6576worker1 # [ 267.624065] nix-grpc-daemon[1875]: time=2026-09-22T15:49:40.226Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6577worker1 # [ 267.626057] nix-grpc-daemon[1875]: time=2026-09-22T15:49:40.226Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6578lb # [ 267.621039] niks3-server[1077]: 2026/09/22 15:49:40 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6579worker1 # [ 267.661913] nix-grpc-daemon[1875]: time=2026-09-22T15:49:40.264Z level=INFO msg="Uploading 1 narinfos"6580lb # [ 267.638812] niks3-server[1077]: 2026/09/22 15:49:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6581lb # [ 267.648594] niks3-server[1077]: 2026/09/22 15:49:40 INFO Signed narinfos id=64 count=16582lb # [ 267.662045] niks3-server[1077]: 2026/09/22 15:49:40 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6583worker1 # [ 267.695876] nix-grpc-daemon[1875]: time=2026-09-22T15:49:40.298Z level=INFO msg="Upload complete. (361ms)"6584lb # [ 267.670573] niks3-server[1077]: 2026/09/22 15:49:40 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6585lb # [ 267.675691] niks3-server[1077]: 2026/09/22 15:49:40 INFO Completed upload id=646586lb # [ 267.679096] niks3-server[1077]: 2026/09/22 15:49:40 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6587worker1 # [ 267.704550] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=5126588worker1 # [ 267.719513] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686589worker1 # [ 267.728202] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6590worker1 # [ 267.730721] nix-daemon[829]: reaped child process 1921, status = succeeded6591worker1 # [ 267.742162] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6592lb # [ 268.554214] envoy[894]: conn peer=[2001:db8:1::1]:43242 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1516 ms=95086593lb # [ 268.559206] envoy[894]: conn peer=[2001:db8:1::3]:54556 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2295 tx=877 ms=1191826594worker1 # [ 268.604732] nix-daemon[829]: reaped child process 1935, status = succeeded6595worker1 # [ 268.747729] nix-daemon[829]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6596worker1 # [ 268.767007] nix-daemon[829]: reaped child process 1954, status = succeeded6597lb # [ 268.750236] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received uploads request method=POST path=/api/pending_closures6598worker1 # [ 268.784426] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6599worker1 # [ 268.786057] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.386Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6600lb # [ 268.787700] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6601worker1 # [ 268.830130] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.432Z level=INFO msg="Uploading 1 narinfos"6602lb # [ 268.805042] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6603lb # [ 268.814840] niks3-server[1077]: 2026/09/22 15:49:41 INFO Signed narinfos id=65 count=16604lb # [ 268.820298] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6605lb # [ 268.827283] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6606lb # [ 268.837693] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6607lb # [ 268.844533] niks3-server[1077]: 2026/09/22 15:49:41 INFO Completed upload id=656608worker1 # [ 268.869333] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.472Z level=INFO msg="Upload complete. (314ms)"6609lb # [ 268.847132] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6610worker2 # [ 268.854921] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:41Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16611worker2 # [ 268.869861] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:41Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06612worker1 # [ 268.881250] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16613worker1 # [ 268.903017] nix-daemon[829]: reaped child process 1933, status = succeeded6614worker1 # [ 268.934630] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686615worker1 # [ 268.950705] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686616worker1 # [ 268.965829] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686617worker1 # [ 268.979924] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6618worker1 # [ 269.135478] nix-daemon[829]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6619worker1 # [ 269.153800] nix-daemon[829]: reaped child process 1961, status = succeeded6620worker1 # [ 269.259548] nix-daemon[829]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6621worker1 # [ 269.274722] nix-daemon[829]: reaped child process 1966, status = succeeded6622lb # [ 269.255762] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received uploads request method=POST path=/api/pending_closures6623worker1 # [ 269.291037] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.893Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6624worker1 # [ 269.292670] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.893Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6625lb # [ 269.281656] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6626lb # [ 269.297883] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6627worker1 # [ 269.329914] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.932Z level=INFO msg="Uploading 1 narinfos"6628lb # [ 269.310089] niks3-server[1077]: 2026/09/22 15:49:41 INFO Signed narinfos id=66 count=16629lb # [ 269.318585] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6630lb # [ 269.332474] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6631worker1 # [ 269.357829] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.960Z level=INFO msg="Upload complete. (362ms)"6632lb # [ 269.342793] niks3-server[1077]: 2026/09/22 15:49:41 INFO Completed upload id=666633worker1 # [ 269.368452] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=5126634lb # [ 269.346392] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6635worker1 # [ 269.379231] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686636worker1 # [ 269.388875] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6637worker1 # [ 269.391859] nix-daemon[829]: reaped child process 1956, status = succeeded6638worker1 # [ 269.403442] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6639worker1 # [ 270.857231] nix-daemon[829]: reaped child process 1970, status = succeeded6640worker1 # [ 270.973464] nix-daemon[829]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)6641worker1 # [ 270.991899] nix-daemon[829]: reaped child process 1989, status = succeeded6642lb # [ 270.970650] niks3-server[1077]: 2026/09/22 15:49:43 INFO Received uploads request method=POST path=/api/pending_closures6643worker1 # [ 271.008444] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.610Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6644worker1 # [ 271.010138] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.610Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6645lb # [ 271.010403] niks3-server[1077]: 2026/09/22 15:49:43 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6646lb # [ 271.021990] niks3-server[1077]: 2026/09/22 15:49:43 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6647lb # [ 271.035486] niks3-server[1077]: 2026/09/22 15:49:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6648worker1 # [ 271.066949] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.669Z level=INFO msg="Uploading 1 narinfos"6649lb # [ 271.042376] niks3-server[1077]: 2026/09/22 15:49:43 INFO Signed narinfos id=67 count=16650lb # [ 271.049615] niks3-server[1077]: 2026/09/22 15:49:43 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6651lb # [ 271.057532] niks3-server[1077]: 2026/09/22 15:49:43 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6652lb # [ 271.067842] niks3-server[1077]: 2026/09/22 15:49:43 INFO Completed upload id=676653worker1 # [ 271.093662] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.696Z level=INFO msg="Upload complete. (277ms)"6654lb # [ 271.071466] niks3-server[1077]: 2026/09/22 15:49:43 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6655worker2 # [ 271.079388] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:43Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16656worker1 # [ 271.108861] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16657worker2 # [ 271.092146] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:43Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06658worker1 # [ 271.126769] nix-daemon[829]: reaped child process 1968, status = succeeded6659worker1 # [ 271.151684] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686660worker1 # [ 271.167091] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686661worker1 # [ 271.181993] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.784Z level=INFO msg="All 1 paths already cached"6662worker1 # [ 271.191665] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686663worker1 # [ 271.205335] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6664worker1 # [ 271.355065] nix-daemon[829]: accepted connection from pid 1993, user nix-grpc-daemon (trusted)6665worker1 # [ 271.381966] nix-daemon[829]: reaped child process 1996, status = succeeded6666worker1 # [ 271.476567] nix-daemon[829]: accepted connection from pid 1998, user nix-grpc-daemon (trusted)6667worker1 # [ 271.495079] nix-daemon[829]: reaped child process 2001, status = succeeded6668lb # [ 271.473162] niks3-server[1077]: 2026/09/22 15:49:44 INFO Received uploads request method=POST path=/api/pending_closures6669worker1 # [ 271.509217] nix-grpc-daemon[1875]: time=2026-09-22T15:49:44.111Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6670worker1 # [ 271.510697] nix-grpc-daemon[1875]: time=2026-09-22T15:49:44.111Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6671lb # [ 271.500662] niks3-server[1077]: 2026/09/22 15:49:44 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6672lb # [ 271.517054] niks3-server[1077]: 2026/09/22 15:49:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6673worker1 # [ 271.550556] nix-grpc-daemon[1875]: time=2026-09-22T15:49:44.153Z level=INFO msg="Uploading 1 narinfos"6674lb # [ 271.529344] niks3-server[1077]: 2026/09/22 15:49:44 INFO Signed narinfos id=68 count=16675lb # [ 271.536423] niks3-server[1077]: 2026/09/22 15:49:44 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6676lb # [ 271.548911] niks3-server[1077]: 2026/09/22 15:49:44 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6677worker1 # [ 271.586103] nix-grpc-daemon[1875]: time=2026-09-22T15:49:44.188Z level=INFO msg="Upload complete. (365ms)"6678lb # [ 271.559675] niks3-server[1077]: 2026/09/22 15:49:44 INFO Completed upload id=686679lb # [ 271.563910] niks3-server[1077]: 2026/09/22 15:49:44 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6680worker1 # [ 271.596384] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=6246681worker1 # [ 271.612253] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686682worker1 # [ 271.619074] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6683worker1 # [ 271.624827] nix-daemon[829]: reaped child process 1991, status = succeeded6684worker1 # [ 271.638262] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6685worker1 # [ 272.924468] nix-daemon[829]: reaped child process 2005, status = succeeded6686worker1 # [ 273.009619] nix-daemon[829]: accepted connection from pid 2021, user nix-grpc-daemon (trusted)6687worker1 # [ 273.028630] nix-daemon[829]: reaped child process 2024, status = succeeded6688lb # [ 273.008827] niks3-server[1077]: 2026/09/22 15:49:45 INFO Received uploads request method=POST path=/api/pending_closures6689worker1 # [ 273.048422] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.650Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6690worker1 # [ 273.049945] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.650Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6691lb # [ 273.054488] niks3-server[1077]: 2026/09/22 15:49:45 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6692lb # [ 273.066525] niks3-server[1077]: 2026/09/22 15:49:45 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6693worker1 # [ 273.104272] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.706Z level=INFO msg="Uploading 1 narinfos"6694lb # [ 273.081451] niks3-server[1077]: 2026/09/22 15:49:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6695lb # [ 273.089621] niks3-server[1077]: 2026/09/22 15:49:45 INFO Signed narinfos id=69 count=16696lb # [ 273.095770] niks3-server[1077]: 2026/09/22 15:49:45 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6697lb # [ 273.107123] niks3-server[1077]: 2026/09/22 15:49:45 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6698lb # [ 273.113042] niks3-server[1077]: 2026/09/22 15:49:45 INFO Completed upload id=696699worker1 # [ 273.139407] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.742Z level=INFO msg="Upload complete. (289ms)"6700lb # [ 273.115850] niks3-server[1077]: 2026/09/22 15:49:45 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6701worker2 # [ 273.123421] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:45Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16702worker1 # [ 273.153263] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16703worker2 # [ 273.135851] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:45Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06704worker1 # [ 273.174882] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686705worker1 # [ 273.182095] nix-daemon[829]: reaped child process 2003, status = succeeded6706worker1 # [ 273.190693] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686707worker1 # [ 273.205185] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.807Z level=INFO msg="All 1 paths already cached"6708worker1 # [ 273.212303] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686709worker1 # [ 273.225819] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6710worker1 # [ 273.387853] nix-daemon[829]: accepted connection from pid 2028, user nix-grpc-daemon (trusted)6711worker1 # [ 273.411306] nix-daemon[829]: reaped child process 2031, status = succeeded6712worker1 # [ 273.517270] nix-daemon[829]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)6713worker1 # [ 273.538377] nix-daemon[829]: reaped child process 2036, status = succeeded6714lb # [ 273.510027] niks3-server[1077]: 2026/09/22 15:49:46 INFO Received uploads request method=POST path=/api/pending_closures6715worker1 # [ 273.547815] nix-grpc-daemon[1875]: time=2026-09-22T15:49:46.150Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6716worker1 # [ 273.549572] nix-grpc-daemon[1875]: time=2026-09-22T15:49:46.150Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6717lb # [ 273.540561] niks3-server[1077]: 2026/09/22 15:49:46 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6718worker1 # [ 273.580253] nix-grpc-daemon[1875]: time=2026-09-22T15:49:46.182Z level=INFO msg="Uploading 1 narinfos"6719lb # [ 273.555354] niks3-server[1077]: 2026/09/22 15:49:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6720lb # [ 273.567448] niks3-server[1077]: 2026/09/22 15:49:46 INFO Signed narinfos id=70 count=16721lb # [ 273.572797] niks3-server[1077]: 2026/09/22 15:49:46 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6722worker1 # [ 273.614286] nix-grpc-daemon[1875]: time=2026-09-22T15:49:46.217Z level=INFO msg="Upload complete. (374ms)"6723lb # [ 273.589414] niks3-server[1077]: 2026/09/22 15:49:46 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6724lb # [ 273.598781] niks3-server[1077]: 2026/09/22 15:49:46 INFO Completed upload id=706725worker1 # [ 273.625456] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=6246726lb # [ 273.601728] niks3-server[1077]: 2026/09/22 15:49:46 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6727worker1 # [ 273.632343] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686728worker1 # [ 273.643488] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6729worker1 # [ 273.648451] nix-daemon[829]: reaped child process 2026, status = succeeded6730worker1 # [ 273.658996] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6731worker1 # [ 275.094417] nix-daemon[829]: reaped child process 2040, status = succeeded6732worker1 # [ 275.194842] nix-daemon[829]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)6733worker1 # [ 275.216603] nix-daemon[829]: reaped child process 2059, status = succeeded6734lb # [ 275.193154] niks3-server[1077]: 2026/09/22 15:49:47 INFO Received uploads request method=POST path=/api/pending_closures6735worker1 # [ 275.232614] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.835Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6736worker1 # [ 275.235727] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.835Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6737lb # [ 275.227999] niks3-server[1077]: 2026/09/22 15:49:47 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6738lb # [ 275.247847] niks3-server[1077]: 2026/09/22 15:49:47 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6739lb # [ 275.262011] niks3-server[1077]: 2026/09/22 15:49:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6740worker1 # [ 275.293252] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.895Z level=INFO msg="Uploading 1 narinfos"6741lb # [ 275.269083] niks3-server[1077]: 2026/09/22 15:49:47 INFO Signed narinfos id=71 count=16742lb # [ 275.275633] niks3-server[1077]: 2026/09/22 15:49:47 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6743lb # [ 275.284252] niks3-server[1077]: 2026/09/22 15:49:47 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6744worker1 # [ 275.314910] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.917Z level=INFO msg="Upload complete. (243ms)"6745lb # [ 275.292112] niks3-server[1077]: 2026/09/22 15:49:47 INFO Completed upload id=716746lb # [ 275.295263] niks3-server[1077]: 2026/09/22 15:49:47 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6747worker2 # [ 275.298155] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:47Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16748worker1 # [ 275.324948] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16749worker1 # [ 275.336439] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686750worker2 # [ 275.311353] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:47Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06751worker1 # [ 275.345861] nix-daemon[829]: reaped child process 2038, status = succeeded6752worker1 # [ 275.353279] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686753worker1 # [ 275.364404] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.967Z level=INFO msg="All 1 paths already cached"6754worker1 # [ 275.372628] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686755worker1 # [ 275.384669] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6756worker1 # [ 275.524728] nix-daemon[829]: accepted connection from pid 2063, user nix-grpc-daemon (trusted)6757worker1 # [ 275.547662] nix-daemon[829]: reaped child process 2066, status = succeeded6758worker1 # [ 275.665870] nix-daemon[829]: accepted connection from pid 2068, user nix-grpc-daemon (trusted)6759worker1 # [ 275.686669] nix-daemon[829]: reaped child process 2071, status = succeeded6760lb # [ 275.663596] niks3-server[1077]: 2026/09/22 15:49:48 INFO Received uploads request method=POST path=/api/pending_closures6761worker1 # [ 275.699750] nix-grpc-daemon[1875]: time=2026-09-22T15:49:48.302Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6762worker1 # [ 275.702213] nix-grpc-daemon[1875]: time=2026-09-22T15:49:48.302Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6763lb # [ 275.694168] niks3-server[1077]: 2026/09/22 15:49:48 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6764worker1 # [ 275.736214] nix-grpc-daemon[1875]: time=2026-09-22T15:49:48.338Z level=INFO msg="Uploading 1 narinfos"6765lb # [ 275.710422] niks3-server[1077]: 2026/09/22 15:49:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6766lb # [ 275.718407] niks3-server[1077]: 2026/09/22 15:49:48 INFO Signed narinfos id=72 count=16767lb # [ 275.725431] niks3-server[1077]: 2026/09/22 15:49:48 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6768worker1 # [ 275.763806] nix-grpc-daemon[1875]: time=2026-09-22T15:49:48.366Z level=INFO msg="Upload complete. (367ms)"6769lb # [ 275.738455] niks3-server[1077]: 2026/09/22 15:49:48 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6770worker1 # [ 275.768264] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=6246771worker1 # [ 275.774038] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686772lb # [ 275.747993] niks3-server[1077]: 2026/09/22 15:49:48 INFO Completed upload id=726773lb # [ 275.750679] niks3-server[1077]: 2026/09/22 15:49:48 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6774worker1 # [ 275.782677] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6775worker1 # [ 275.783925] nix-daemon[829]: reaped child process 2061, status = succeeded6776worker1 # [ 275.791094] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6777worker1 # [ 276.425086] nix-daemon[829]: reaped child process 2075, status = succeeded6778worker1 # [ 276.541243] nix-daemon[829]: accepted connection from pid 2091, user nix-grpc-daemon (trusted)6779worker1 # [ 276.559293] nix-daemon[829]: reaped child process 2094, status = succeeded6780lb # [ 276.539043] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received uploads request method=POST path=/api/pending_closures6781worker1 # [ 276.577690] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.179Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6782worker1 # [ 276.579152] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.179Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6783worker1 # [ 276.622187] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.224Z level=INFO msg="Uploading 1 narinfos"6784lb # [ 276.584507] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6785lb # [ 276.600167] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6786lb # [ 276.612333] niks3-server[1077]: 2026/09/22 15:49:49 INFO Signed narinfos id=73 count=16787lb # [ 276.619036] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6788lb # [ 276.627599] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6789worker1 # [ 276.664750] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.267Z level=INFO msg="Upload complete. (264ms)"6790lb # [ 276.639091] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6791lb # [ 276.642988] niks3-server[1077]: 2026/09/22 15:49:49 INFO Completed upload id=736792lb # [ 276.647283] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6793worker2 # [ 276.648662] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:49Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16794worker1 # [ 276.677322] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16795worker1 # [ 276.698583] nix-daemon[829]: reaped child process 2073, status = succeeded6796worker1 # [ 276.720808] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686797worker2 # [ 276.698505] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:49Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06798worker1 # [ 276.736639] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686799worker1 # [ 276.750679] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.353Z level=INFO msg="All 2 paths already cached"6800worker1 # [ 276.760476] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686801worker1 # [ 276.773105] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6802worker1 # [ 276.922953] nix-daemon[829]: accepted connection from pid 2098, user nix-grpc-daemon (trusted)6803worker1 # [ 276.943725] nix-daemon[829]: reaped child process 2101, status = succeeded6804worker1 # [ 277.042412] nix-daemon[829]: accepted connection from pid 2103, user nix-grpc-daemon (trusted)6805worker1 # [ 277.058908] nix-daemon[829]: reaped child process 2106, status = succeeded6806lb # [ 277.037817] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received uploads request method=POST path=/api/pending_closures6807worker1 # [ 277.074321] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.676Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6808worker1 # [ 277.075814] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.677Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6809lb # [ 277.065219] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6810worker1 # [ 277.105640] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.708Z level=INFO msg="Uploading 1 narinfos"6811lb # [ 277.079891] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6812lb # [ 277.091459] niks3-server[1077]: 2026/09/22 15:49:49 INFO Signed narinfos id=74 count=16813lb # [ 277.098059] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6814worker1 # [ 277.137465] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.740Z level=INFO msg="Upload complete. (348ms)"6815lb # [ 277.111512] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6816lb # [ 277.120220] niks3-server[1077]: 2026/09/22 15:49:49 INFO Completed upload id=746817worker1 # [ 277.145565] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=7526818lb # [ 277.122927] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6819worker1 # [ 277.158609] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686820worker1 # [ 277.166213] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6821worker1 # [ 277.172292] nix-daemon[829]: reaped child process 2096, status = succeeded6822worker1 # [ 277.181030] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6823worker1 # [ 278.456193] nix-daemon[829]: reaped child process 2110, status = succeeded6824worker1 # [ 278.561196] nix-daemon[829]: accepted connection from pid 2126, user nix-grpc-daemon (trusted)6825worker1 # [ 278.581580] nix-daemon[829]: reaped child process 2129, status = succeeded6826lb # [ 278.562171] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received uploads request method=POST path=/api/pending_closures6827worker1 # [ 278.600366] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.200Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6828worker1 # [ 278.601858] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.200Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6829lb # [ 278.592807] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6830lb # [ 278.608957] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6831lb # [ 278.624695] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6832worker1 # [ 278.657854] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.260Z level=INFO msg="Uploading 1 narinfos"6833lb # [ 278.635753] niks3-server[1077]: 2026/09/22 15:49:51 INFO Signed narinfos id=75 count=16834lb # [ 278.641485] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6835lb # [ 278.652893] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6836lb # [ 278.659962] niks3-server[1077]: 2026/09/22 15:49:51 INFO Completed upload id=756837worker1 # [ 278.686812] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.289Z level=INFO msg="Upload complete. (295ms)"6838lb # [ 278.665353] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6839worker2 # [ 278.673513] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:51Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16840worker1 # [ 278.700735] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16841worker1 # [ 278.722075] nix-daemon[829]: reaped child process 2108, status = succeeded6842worker1 # [ 278.746718] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686843worker1 # [ 278.762086] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686844worker1 # [ 278.777798] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.380Z level=INFO msg="All 2 paths already cached"6845worker1 # [ 278.787909] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686846worker1 # [ 278.802165] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6847worker1 # [ 278.956096] nix-daemon[829]: accepted connection from pid 2133, user nix-grpc-daemon (trusted)6848worker1 # [ 278.976148] nix-daemon[829]: reaped child process 2136, status = succeeded6849worker1 # [ 279.078588] nix-daemon[829]: accepted connection from pid 2138, user nix-grpc-daemon (trusted)6850worker1 # [ 279.093997] nix-daemon[829]: reaped child process 2141, status = succeeded6851lb # [ 279.074561] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received uploads request method=POST path=/api/pending_closures6852worker1 # [ 279.110902] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.713Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6853worker1 # [ 279.112446] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.713Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6854lb # [ 279.101672] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6855lb # [ 279.118527] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6856worker1 # [ 279.151912] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.754Z level=INFO msg="Uploading 1 narinfos"6857lb # [ 279.131618] niks3-server[1077]: 2026/09/22 15:49:51 INFO Signed narinfos id=76 count=16858lb # [ 279.136632] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6859lb # [ 279.147964] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6860lb # [ 279.153424] niks3-server[1077]: 2026/09/22 15:49:51 INFO Completed upload id=766861worker1 # [ 279.179011] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.781Z level=INFO msg="Upload complete. (360ms)"6862lb # [ 279.155585] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6863worker1 # [ 279.189068] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=7526864worker1 # [ 279.199651] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686865worker1 # [ 279.206607] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6866worker1 # [ 279.211350] nix-daemon[829]: reaped child process 2131, status = succeeded6867worker1 # [ 279.224317] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6868worker1 # [ 280.704161] nix-daemon[829]: reaped child process 2145, status = succeeded6869worker1 # [ 280.808114] nix-daemon[829]: accepted connection from pid 2161, user nix-grpc-daemon (trusted)6870worker1 # [ 280.824106] nix-daemon[829]: reaped child process 2164, status = succeeded6871lb # [ 280.806830] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received uploads request method=POST path=/api/pending_closures6872worker1 # [ 280.844423] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.446Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6873worker1 # [ 280.846593] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.446Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6874lb # [ 280.843302] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6875lb # [ 280.857729] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6876worker1 # [ 280.897915] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.500Z level=INFO msg="Uploading 1 narinfos"6877lb # [ 280.871784] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6878lb # [ 280.879541] niks3-server[1077]: 2026/09/22 15:49:53 INFO Signed narinfos id=77 count=16879lb # [ 280.884822] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6880lb # [ 280.893707] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6881lb # [ 280.899721] niks3-server[1077]: 2026/09/22 15:49:53 INFO Completed upload id=776882worker1 # [ 280.926064] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.528Z level=INFO msg="Upload complete. (290ms)"6883lb # [ 280.902090] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6884worker2 # [ 280.909818] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:53Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16885worker1 # [ 280.941261] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16886worker1 # [ 280.960366] nix-daemon[829]: reaped child process 2143, status = succeeded6887worker1 # [ 280.986598] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686888worker1 # [ 281.001998] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686889worker1 # [ 281.016636] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.619Z level=INFO msg="All 2 paths already cached"6890worker1 # [ 281.026814] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500686891worker1 # [ 281.040580] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6892worker1 # [ 281.197472] nix-daemon[829]: accepted connection from pid 2168, user nix-grpc-daemon (trusted)6893worker1 # [ 281.218626] nix-daemon[829]: reaped child process 2171, status = succeeded6894worker1 # [ 281.332977] nix-daemon[829]: accepted connection from pid 2173, user nix-grpc-daemon (trusted)6895worker1 # [ 281.350513] nix-daemon[829]: reaped child process 2176, status = succeeded6896lb # [ 281.328281] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received uploads request method=POST path=/api/pending_closures6897worker1 # [ 281.364505] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.967Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6898worker1 # [ 281.365995] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.967Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6899lb # [ 281.355184] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6900worker1 # [ 281.394255] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.997Z level=INFO msg="Uploading 1 narinfos"6901lb # [ 281.371104] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6902lb # [ 281.380688] niks3-server[1077]: 2026/09/22 15:49:53 INFO Signed narinfos id=78 count=16903lb # [ 281.391840] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6904worker1 # [ 281.427291] nix-grpc-daemon[1875]: time=2026-09-22T15:49:54.030Z level=INFO msg="Upload complete. (370ms)"6905lb # [ 281.402191] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6906lb # [ 281.407676] niks3-server[1077]: 2026/09/22 15:49:53 INFO Completed upload id=786907worker1 # [ 281.433432] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=7526908lb # [ 281.411211] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6909worker1 # [ 281.444280] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500686910worker1 # [ 281.454193] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6911worker1 # [ 281.458840] nix-daemon[829]: reaped child process 2166, status = succeeded6912worker1 # [ 281.469206] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6913worker1 # [ 282.575950] nix-daemon[829]: reaped child process 2180, status = succeeded6914worker1 # [ 282.643970] nix-daemon[829]: accepted connection from pid 2196, user nix-grpc-daemon (trusted)6915worker1 # [ 282.662204] nix-daemon[829]: reaped child process 2199, status = succeeded6916lb # [ 282.643630] niks3-server[1077]: 2026/09/22 15:49:55 INFO Received uploads request method=POST path=/api/pending_closures6917worker1 # [ 282.683066] nix-grpc-daemon[1875]: time=2026-09-22T15:49:55.284Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6918worker1 # [ 282.684960] nix-grpc-daemon[1875]: time=2026-09-22T15:49:55.284Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6919lb # [ 282.683654] niks3-server[1077]: 2026/09/22 15:49:55 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6920lb # [ 282.696787] niks3-server[1077]: 2026/09/22 15:49:55 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6921worker1 # [ 282.734498] nix-grpc-daemon[1875]: time=2026-09-22T15:49:55.337Z level=INFO msg="Uploading 1 narinfos"6922lb # [ 282.710973] niks3-server[1077]: 2026/09/22 15:49:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6923lb # [ 282.719771] niks3-server[1077]: 2026/09/22 15:49:55 INFO Signed narinfos id=79 count=16924lb # [ 282.726163] niks3-server[1077]: 2026/09/22 15:49:55 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6925lb # [ 282.732907] niks3-server[1077]: 2026/09/22 15:49:55 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6926lb # [ 282.742110] niks3-server[1077]: 2026/09/22 15:49:55 INFO Completed upload id=796927worker1 # [ 282.767894] nix-grpc-daemon[1875]: time=2026-09-22T15:49:55.370Z level=INFO msg="Upload complete. (286ms)"6928lb # [ 282.745567] niks3-server[1077]: 2026/09/22 15:49:55 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6929worker1 # [ 282.781236] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=16930worker1 # [ 282.799905] nix-daemon[829]: reaped child process 2178, status = succeeded6931client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.74 seconds)6932worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6933worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6934client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6935client # [ 283.003077] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6936client # Running as unit: gpu.service; invocation ID: 515d8eac035444d59343ce9b425311b16937client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6938lb: must succeed: curl -sf localhost:9901/clusters6939lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6940lb: must succeed: curl -sf localhost:9901/clusters6941lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6942worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6943worker2 # [ 283.206764] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377106944client # [ 283.221260] nix[1949]: don't know how to build these paths:6945client # [ 283.222419] nix[1949]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6946worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6947worker1 # [ 283.245685] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500686948worker2 # [ 283.244257] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:55Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06949client # [ 283.257618] nix[1949]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6950lb: must succeed: curl -sf localhost:9901/clusters6951lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6952lb: must succeed: curl -sf localhost:9901/clusters6953lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6954worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6955worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6956client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6957lb # [ 288.560538] envoy[894]: conn peer=[2001:db8:1::1]:41934 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27118 tx=12410 ms=183626958client # [ 313.264708] nix[1949]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6959client # [ 313.279773] nix[1949]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6960client # [ 313.314169] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6961client # [ 313.320964] systemd[1]: gpu.service: Failed with result 'exit-code'.6962client # [ 313.324727] systemd[1]: gpu.service: Consumed 195ms CPU time over 30.310s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6963client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.09 seconds)6964client: must succeed: journalctl -u gpu -o cat6965client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6966lb: must succeed: curl -sf localhost:9901/clusters6967lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6968lb: must succeed: curl -sf localhost:9901/clusters6969lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6970worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6971worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6972(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.59 seconds)6973subtest: low disk drains a worker and builds go to the other6974worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6975worker1 # [ 314.869216] nix-grpc-daemon[1815]: ts=2026-09-22T15:50:27Z level=info event=unhealthy reason=min_free6976worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.20 seconds)6977worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6978worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.03 seconds)6979client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26980worker1 # [ 315.189540] nix-grpc-daemon[1815]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500686981client # don't know how to build these paths:6982client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6983worker2 # [ 315.177334] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377106984worker2 # [ 315.202872] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06985worker2 # [ 315.210362] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16986worker2 # [ 315.248987] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377106987worker2 # [ 315.265165] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377106988client # copying 1 paths to 'lb:50051'...6989client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6990worker2 # [ 315.281239] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377106991worker2 # [ 315.298015] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6992worker2 # [ 315.456195] nix-daemon[856]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)6993worker2 # [ 315.476172] nix-daemon[856]: reaped child process 1978, status = succeeded6994worker2 # [ 315.577724] nix-daemon[856]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)6995worker2 # [ 315.595701] nix-daemon[856]: reaped child process 1983, status = succeeded6996lb # [ 315.602436] niks3-server[1077]: 2026/09/22 15:50:28 INFO Received uploads request method=POST path=/api/pending_closures6997worker2 # [ 315.616275] nix-grpc-daemon[1754]: time=2026-09-22T15:50:28.215Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6998worker2 # [ 315.617801] nix-grpc-daemon[1754]: time=2026-09-22T15:50:28.215Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6999lb # [ 315.636856] niks3-server[1077]: 2026/09/22 15:50:28 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7000worker2 # [ 315.650364] nix-grpc-daemon[1754]: time=2026-09-22T15:50:28.249Z level=INFO msg="Uploading 1 narinfos"7001lb # [ 315.650417] niks3-server[1077]: 2026/09/22 15:50:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7002lb # [ 315.660176] niks3-server[1077]: 2026/09/22 15:50:28 INFO Signed narinfos id=80 count=17003lb # [ 315.668034] niks3-server[1077]: 2026/09/22 15:50:28 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7004lb # [ 315.685798] niks3-server[1077]: 2026/09/22 15:50:28 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7005lb # [ 315.692241] niks3-server[1077]: 2026/09/22 15:50:28 INFO Completed upload id=807006worker2 # [ 315.691367] nix-grpc-daemon[1754]: time=2026-09-22T15:50:28.290Z level=INFO msg="Upload complete. (378ms)"7007lb # [ 315.700329] niks3-server[1077]: 2026/09/22 15:50:28 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7008worker2 # [ 315.700843] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=5127009worker2 # [ 315.716551] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377107010worker2 # [ 315.726593] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)7011worker2 # [ 315.735650] nix-daemon[856]: reaped child process 1973, status = succeeded7012client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7013worker2 # [ 315.743266] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)7014client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7015lb # [ 316.011666] postgres[798]: [798] LOG: checkpoint starting: time7016client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7017client # farm-a> LOG-farm-a-drain7018worker2 # [ 316.436258] nix-daemon[856]: reaped child process 1987, status = succeeded7019worker2 # [ 316.504289] nix-daemon[856]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)7020worker2 # [ 316.521155] nix-daemon[856]: reaped child process 2006, status = succeeded7021lb # [ 316.525710] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received uploads request method=POST path=/api/pending_closures7022worker2 # [ 316.536090] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7023worker2 # [ 316.537721] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.134Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7024lb # [ 316.568627] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7025lb # [ 316.582119] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7026worker2 # [ 316.590807] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.189Z level=INFO msg="Uploading 1 narinfos"7027lb # [ 316.593119] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7028lb # [ 316.602240] niks3-server[1077]: 2026/09/22 15:50:29 INFO Signed narinfos id=81 count=17029lb # [ 316.608977] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7030lb # [ 316.615785] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7031lb # [ 316.624528] niks3-server[1077]: 2026/09/22 15:50:29 INFO Completed upload id=817032worker2 # [ 316.629401] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.222Z level=INFO msg="Upload complete. (268ms)"7033lb # [ 316.629314] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7034worker2 # [ 316.641844] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=17035worker2 # [ 316.662512] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07036worker2 # [ 316.668539] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17037worker2 # [ 316.674712] nix-daemon[856]: reaped child process 1985, status = succeeded7038worker2 # [ 316.709101] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377107039worker2 # [ 316.724473] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377107040worker2 # [ 316.739931] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.339Z level=INFO msg="All 1 paths already cached"7041client # copying 1 paths to 'lb:50051'...7042client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7043worker2 # [ 316.751063] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377107044worker2 # [ 316.765838] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)7045worker2 # [ 316.918716] nix-daemon[856]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)7046worker2 # [ 316.938294] nix-daemon[856]: reaped child process 2015, status = succeeded7047worker2 # [ 317.038710] nix-daemon[856]: accepted connection from pid 2017, user nix-grpc-daemon (trusted)7048worker2 # [ 317.057453] nix-daemon[856]: reaped child process 2020, status = succeeded7049lb # [ 317.054283] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received uploads request method=POST path=/api/pending_closures7050worker2 # [ 317.065252] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.664Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7051worker2 # [ 317.066792] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.664Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7052lb # [ 317.085352] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7053worker2 # [ 317.095815] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.695Z level=INFO msg="Uploading 1 narinfos"7054lb # [ 317.098416] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7055lb # [ 317.108667] niks3-server[1077]: 2026/09/22 15:50:29 INFO Signed narinfos id=82 count=17056lb # [ 317.114163] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7057lb # [ 317.123554] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7058worker2 # [ 317.128631] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.727Z level=INFO msg="Upload complete. (346ms)"7059lb # [ 317.129312] niks3-server[1077]: 2026/09/22 15:50:29 INFO Completed upload id=827060lb # [ 317.131691] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7061worker2 # [ 317.136233] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=6327062worker2 # [ 317.150019] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377107063worker2 # [ 317.155627] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)7064worker2 # [ 317.160669] nix-daemon[856]: reaped child process 2010, status = succeeded7065client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7066worker2 # [ 317.172616] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)7067client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7068client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7069lb # [ 318.573042] envoy[894]: conn peer=[2001:db8:1::1]:51474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=580 ms=300767070client # farm-b> LOG-farm-b-drain7071worker2 # [ 318.882565] nix-daemon[856]: reaped child process 2024, status = succeeded7072worker2 # [ 318.956300] nix-daemon[856]: accepted connection from pid 2040, user nix-grpc-daemon (trusted)7073worker2 # [ 318.975886] nix-daemon[856]: reaped child process 2043, status = succeeded7074lb # [ 318.979139] niks3-server[1077]: 2026/09/22 15:50:31 INFO Received uploads request method=POST path=/api/pending_closures7075worker2 # [ 318.992555] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.590Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7076worker2 # [ 318.994775] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.590Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7077lb # [ 319.021891] niks3-server[1077]: 2026/09/22 15:50:31 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7078lb # [ 319.035429] niks3-server[1077]: 2026/09/22 15:50:31 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7079worker2 # [ 319.049955] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.649Z level=INFO msg="Uploading 1 narinfos"7080lb # [ 319.050653] niks3-server[1077]: 2026/09/22 15:50:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7081lb # [ 319.059282] niks3-server[1077]: 2026/09/22 15:50:31 INFO Signed narinfos id=83 count=17082lb # [ 319.066928] niks3-server[1077]: 2026/09/22 15:50:31 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7083worker2 # [ 319.075280] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.674Z level=INFO msg="Upload complete. (291ms)"7084lb # [ 319.076809] niks3-server[1077]: 2026/09/22 15:50:31 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7085lb # [ 319.084475] niks3-server[1077]: 2026/09/22 15:50:31 INFO Completed upload id=837086lb # [ 319.087006] niks3-server[1077]: 2026/09/22 15:50:31 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7087worker2 # [ 319.089138] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=17088worker2 # [ 319.112353] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07089worker2 # [ 319.118282] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17090worker2 # [ 319.123272] nix-daemon[856]: reaped child process 2022, status = succeeded7091worker2 # [ 319.153378] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377107092worker2 # [ 319.168743] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377107093worker2 # [ 319.182507] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.781Z level=INFO msg="All 2 paths already cached"7094client # copying 1 paths to 'lb:50051'...7095client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7096worker2 # [ 319.193363] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377107097worker2 # [ 319.208834] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)7098worker2 # [ 319.362343] nix-daemon[856]: accepted connection from pid 2049, user nix-grpc-daemon (trusted)7099worker2 # [ 319.381741] nix-daemon[856]: reaped child process 2052, status = succeeded7100worker2 # [ 319.486983] nix-daemon[856]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)7101worker2 # [ 319.503728] nix-daemon[856]: reaped child process 2057, status = succeeded7102lb # [ 319.508952] niks3-server[1077]: 2026/09/22 15:50:32 INFO Received uploads request method=POST path=/api/pending_closures7103worker2 # [ 319.518370] nix-grpc-daemon[1754]: time=2026-09-22T15:50:32.117Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7104worker2 # [ 319.519920] nix-grpc-daemon[1754]: time=2026-09-22T15:50:32.117Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7105lb # [ 319.535057] niks3-server[1077]: 2026/09/22 15:50:32 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7106worker2 # [ 319.551704] nix-grpc-daemon[1754]: time=2026-09-22T15:50:32.150Z level=INFO msg="Uploading 1 narinfos"7107lb # [ 319.551594] niks3-server[1077]: 2026/09/22 15:50:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7108lb # [ 319.560956] niks3-server[1077]: 2026/09/22 15:50:32 INFO Signed narinfos id=84 count=17109lb # [ 319.563657] niks3-server[1077]: 2026/09/22 15:50:32 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7110worker2 # [ 319.580232] nix-grpc-daemon[1754]: time=2026-09-22T15:50:32.179Z level=INFO msg="Upload complete. (356ms)"7111lb # [ 319.580784] niks3-server[1077]: 2026/09/22 15:50:32 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7112lb # [ 319.588633] niks3-server[1077]: 2026/09/22 15:50:32 INFO Completed upload id=847113lb # [ 319.592539] niks3-server[1077]: 2026/09/22 15:50:32 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7114worker2 # [ 319.588243] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=7607115worker2 # [ 319.601453] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377107116worker2 # [ 319.608249] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)7117worker2 # [ 319.613022] nix-daemon[856]: reaped child process 2047, status = succeeded7118client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7119worker2 # [ 319.626646] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)7120client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7121client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7122client # farm-top> LOG-farm-top-drain7123worker2 # [ 320.885647] nix-daemon[856]: reaped child process 2061, status = succeeded7124worker2 # [ 320.983324] nix-daemon[856]: accepted connection from pid 2077, user nix-grpc-daemon (trusted)7125worker2 # [ 321.002191] nix-daemon[856]: reaped child process 2080, status = succeeded7126lb # [ 321.009549] niks3-server[1077]: 2026/09/22 15:50:33 INFO Received uploads request method=POST path=/api/pending_closures7127worker2 # [ 321.020418] nix-grpc-daemon[1754]: time=2026-09-22T15:50:33.618Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7128worker2 # [ 321.022090] nix-grpc-daemon[1754]: time=2026-09-22T15:50:33.618Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7129lb # [ 321.051107] niks3-server[1077]: 2026/09/22 15:50:33 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7130lb # [ 321.065148] niks3-server[1077]: 2026/09/22 15:50:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7131worker2 # [ 321.071492] nix-grpc-daemon[1754]: time=2026-09-22T15:50:33.670Z level=INFO msg="Uploading 1 narinfos"7132lb # [ 321.078053] niks3-server[1077]: 2026/09/22 15:50:33 INFO Signed narinfos id=85 count=17133lb # [ 321.083818] niks3-server[1077]: 2026/09/22 15:50:33 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7134lb # [ 321.095099] niks3-server[1077]: 2026/09/22 15:50:33 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7135lb # [ 321.106784] niks3-server[1077]: 2026/09/22 15:50:33 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7136worker2 # [ 321.114732] nix-grpc-daemon[1754]: time=2026-09-22T15:50:33.714Z level=INFO msg="Upload complete. (278ms)"7137lb # [ 321.114852] niks3-server[1077]: 2026/09/22 15:50:33 INFO Completed upload id=857138lb # [ 321.120074] niks3-server[1077]: 2026/09/22 15:50:33 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7139worker2 # [ 321.129227] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=17140worker2 # [ 321.149770] nix-daemon[856]: reaped child process 2059, status = succeeded7141client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.23 seconds)7142client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7143client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7144worker1: must succeed: rm /nix/.rw-store/fill7145worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7146worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7147worker1 # [ 321.873309] nix-grpc-daemon[1815]: ts=2026-09-22T15:50:34Z level=info event=healthy reason=min_free7148worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7149(finished: subtest: low disk drains a worker and builds go to the other, in 8.99 seconds)7150(finished: run the VM test script, in 323.63 seconds)7151test script finished in 323.90s7152cleanup7153kill QemuMachine (pid 49)7154client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7155client # [2026-09-22T15:50:35Z INFO virtiofsd] Client disconnected, shutting down7156client # [2026-09-22T15:50:35Z INFO virtiofsd] Client disconnected, shutting down7157client # [2026-09-22T15:50:35Z INFO virtiofsd] Client disconnected, shutting down7158kill QemuMachine (pid 50)7159lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7160lb # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7161lb # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7162lb # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7163kill QemuMachine (pid 51)7164worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7165worker1 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7166worker1 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7167worker1 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7168kill QemuMachine (pid 52)7169worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7170worker2 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7171worker2 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7172worker2 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down7173(finished: cleanup, in 1.46 seconds)7174additionally exposed symbols:7175 client, lb, worker1, worker2,7176 vlan1,7177 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh